Fix path handling in download-mongodb #498
Conversation
|
Worst case I'll follow up with an inline retry function because we're already seeing a failure with: |
|
test-oidc-k8s-local failure is expected, I had to tear down the cluster today |
.evergreen/download-mongodb.sh
Outdated
| if [ -n "${MONGODB_BINARIES:-}" ]; then | ||
| cd "$(dirname "$(dirname "${MONGODB_BINARIES:?}")")" | ||
| else | ||
| cd $DRIVERS_TOOLS | ||
| fi | ||
|
|
||
| echo "Installing server binaries..." | ||
| "$SCRIPT_DIR/retry-with-backoff.sh" curl $MONGODB_DOWNLOAD_URL --output mongodb-binaries.tgz | ||
| curl $MONGODB_DOWNLOAD_URL --output mongodb-binaries.tgz |
There was a problem hiding this comment.
We should be using curl_retry here. curl_retry is defined in this file. Same for all the other curl calls.
There was a problem hiding this comment.
Oh you removed curl_retry in PYTHON-4734. How about we just revert that commit entirely (again)?
There was a problem hiding this comment.
retry-with-backoff.sh
Has curl's native command line flags been considered?
--retry <num>If a transient error is returned when curl tries to perform a transfer, it will retry this number of times before giving up. Setting the number to 0 makes curl do no retries (which is the default). Transient error means either: a timeout, an FTP 4xx response code or an HTTP 408, 429, 500, 502, 503 or 504 response code.
When curl is about to retry a transfer, it will first wait one second and then for all forthcoming retries it will double the waiting time until it reaches 10 minutes which then will be the delay between the rest of the retries.
This behavior seems completely equivalent to the emulated behavior of the retry-with-backoff.sh script. See also --max-time, --retry-max-time, --retry-connrefused, and --retry-all-errors.
|
We previously used curl's built in retries but not all curls we use are new enough because we test on so many different/old hosts. See PYTHON-3721 and DRIVERS-2625. |
|
Updated to restore |
Co-authored-by: Ezra Chung <[email protected]>
Co-authored-by: Ezra Chung <[email protected]>
I tested this by cloning the c driver and invoking the integration script. The way that script gets invoked is pretty gnarly and unexpected. The path ends up being
.evergreen/scripts/../../drivers-evergreen-tools/.evergreen/download-mongodb.sh, and I'm not sure how to get that into a suitable path. I'll have to follow up with a more robust solution.