Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The download function ignores the overwrite attribute and crashes on downloading the thumbnail #3

Open
stpCES opened this issue Sep 18, 2022 · 2 comments

Comments

@stpCES
Copy link

stpCES commented Sep 18, 2022

items = download_aws(assets="overview",
+                      bbox = bbox,
+                      after = "2022-09-02",
+                      before = "2022-09-04",
+                      timeframe = "full",
+                      max.cloud = 30, outdir = "C:\\SOM-Togdheer\\R-programs",
+                      overwrite = TRUE,
+                      verbose = T,
+                      query_only = F,
+                      )
STAC version: 1.0.0-beta.2
Querying data for temporal window 2022-09-02/2022-09-04
The following items matched the specified spatiotemporal extent.
###STACItemCollection
- matched feature(s): 4
- features (4 item(s) / 0 not fetched):
  - S2A_38PMR_20220904_0_L2A
  - S2A_38PNR_20220904_0_L2A
  - S2A_38PMS_20220904_0_L2A
  - S2A_38PNS_20220904_0_L2A
- assets: 
thumbnail, overview, info, metadata, visual, B01, B02, B03, B04, B05, B06, B07, B08, B8A, B09, B11, B12, AOT, WVP, SCL
- other field(s): 
type, stac_version, stac_extensions, context, numberMatched, numberReturned, features, links
Starting download...
  |=========================                         |  50%Error: Error while downloading https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/38/P/NR/2022/9/S2A_38PNR_20220904_0_L2A/L2A_PVI.tif. 
Path exists and overwrite is FALSE

It actually downloads the file and afterwards says it already existing and overwrite would be false even though I specified it to be true. I changed the outdir in case it would have something to do with Windows access permissions but it doesn't change anything.

Moreover the function also interrupts when trying to download the all assets because the thumbnail is first and it gives the following error

Starting download...
  |============                                      |  25%Error: Error while downloading https://roda.sentinel-hub.com/sentinel-s2-l1c/tiles/38/P/MR/2022/9/4/0/preview.jpg. 
Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.
@stpCES stpCES changed the title The download function irgnores the overwrite attribute and crashes on downloading the thumbnail The download function ignores the overwrite attribute and crashes on downloading the thumbnail Sep 18, 2022
@goergen95
Copy link
Member

Thanks! I see the same issue when R's internal downloader is used. I am unsure when I will be able to fix this (at least about 2 weeks). Please consider downloading the aria2c executable for your system, set use_aria = TRUE and point aria_bin towards where you installed the executable. This comes with the additional benefit of parallel downloads thus speeding up the whole process. Also consider if your really need to download all assets. For example, the visual asset is simply a RBG-composite of Band 02, 03, 04 thus this data would be downloaded twice.

@stpCES
Copy link
Author

stpCES commented Sep 18, 2022

Yes, I started the download using the aria2c now. Let's see if it works for the whole data package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants