Using nextclade dataset get behind proxy gives CURLE_SSL_CONNECT_ERROR

[stefandiederich]

Hi,
I am using NextClade version 1.3.0 with the Docker image.
Now I am stuck at the point where I have to download the dataset with
docker run -it --rm -u 1001 nextstrain/nextclade nextclade dataset get --name='sars-cov-2' --output-dir='data/sars-cov-2'

It tells me the following error:

[ERROR] Nextclade: When fetching a file "https://data.clades.nextstrain.org/index.json": CURLE_SSL_CONNECT_ERROR: SSL connect error. Please verify the correctness of the address, check your internet connection and try again later. Make sure that you have a permission to access the requested resource.

Because we are working behind a proxy server I tried to hand in the proxy config with the -e parameter like this

docker run -it --rm -u 1001 -e "HTTP_PROXY=http://<IP>:<PORT>/" -e "HTTPS_PROXY=https://<IP>:<PORT>/" nextstrain/nextclade nextclade dataset get --name='sars-cov-2' --output-dir='data/sars-cov-2'

But the error persists. Did I do anything wrong or is the error not proxy related?
Bests
Stefan

[ivan-aksamentov]

Hi @stefandiederich,

We haven't thought about it.

nextclade dataset commands use libcurl underneath. Can you try to use curl to fetch the same index.json file and see if it works?

curl --compressed https://data.clades.nextstrain.org/index.json

or in container:

docker run -it --rm -u 1001 nextstrain/nextclade curl --compressed https://data.clades.nextstrain.org/index.json

Is https://data.clades.nextstrain.org/index.json reachable in your browser?

[stefandiederich]

Hi @ivan-aksamentov ,
thanks for that quick reply.
The json file is reachable via browser.
With curl -s -x https://<IP>:<PORT> https://data.clades.nextstrain.org/index.json > index.json it takes a few minutes and completes without error giving me an empty file.

[stefandiederich]

With docker run -it --rm -u 1001 nextstrain/nextclade curl --compressed https://data.clades.nextstrain.org/index.json I get the following error message curl: (7) Failed to connect to data.clades.nextstrain.org port 443: Network is unreachable

[ivan-aksamentov]

If curl does not work, there isn't much I can do, I'm afraid. Can you contact your admin for help?

[stefandiederich]

I am not sure why curl is not working. I hve used it many times before... I will try to get it work and come back to you.
Is there a way to download the files via browser?

[ivan-aksamentov]

Is there a way to download the files via browser?

Yes, but it's very cumbersome. If you can see the index.json file in the browser, you can find file property in each dataset, with a list of file URLs. If you append these relative URLs to https://data.clades.nextstrain.org/, this will give fill file URLs. Download all files in a directory and it will be the dataset.

[stefandiederich]

Okay, now I curl is working. I had to specifiy the http proxy for downloading the https files... thougth that should be also https proxy then...

[stefandiederich]

So is there a way to easily download via curl?

[corneliusroemer]

This comment may help: #726 (comment)