Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Retry mechanism for transient errors #49

Merged
merged 7 commits into from
Jul 3, 2024
Merged

Retry mechanism for transient errors #49

merged 7 commits into from
Jul 3, 2024

Commits on Jun 29, 2024

  1. Remove transient error code in test_fetch_url_http_error

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    a014b00 View commit details
    Browse the repository at this point in the history

  2. Retry up to 5 times when encountering a transient error

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    b27d5cf View commit details
    Browse the repository at this point in the history

  3. Test transient error retrying

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    da1ce78 View commit details
    Browse the repository at this point in the history

  4. Moved TRANSIENT_ERRORS global variable outside of function

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    66c629d View commit details
    Browse the repository at this point in the history

  5. Made max retry attempts configurable

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    db52f22 View commit details
    Browse the repository at this point in the history

  6. Add test for custom amount of retry attempts

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    49029a2 View commit details
    Browse the repository at this point in the history

  7. Add tests for transient error retrying in crawl method

    @Mews
    Mews committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    1fe33fa View commit details
    Browse the repository at this point in the history