Downloading SicHub offline images

General Discussions
1

Dear Sen4Cap,

We are aware that SciHub keeps data for maximum a year and it’s archived after that. Get to request if we want to get data from archive. My question is there is any way we can resume downloading from sen4cap after SciHub gives access to requested images.

Thanks and Regards,
Henry

2

Dear Henry,

The system is already handling this (we saw users successfully downloading from SciHub products from 2018).
But … there are some issues here, that you should take into account:

  1. There is not the issue that the products become available after several hours after the request was done to the archive. The real issue is with the quota that SciHub also implements (the number of requests that can be done in an hour and per day). After this quota is reached, SciHub will just return 403.
  2. The mechanism is working if you have small sets of data (small seasons and site extents for 1-2 tiles). Otherwise, it will take forever to download all the products.
  3. The system can be configured to retry for a long time for products to download. By default, the system will try to download a product each hour (“Retry interval minutes” = 60 minutes), for 3 days (“Max retries” = 72). After that, it will give up. For the archive mode, you must increase the number of retries (there should be no need to change also the retry interval as it will not matter in this mode) for the datasource by putting Max retries = 9000 (this means retry for one year for a product …). But if you want to use this mode, you should be very patient …

Another solution would be to move on a DIAS where normally, at least in this moment, it is not this restriction.

Best regards,
Cosmin

1 Like
3

Dear Cosmin,

Thank you for your explanation. Never tried DIAS before. I will look into it.

I noticed that there’s downloader_history table in postgres and wonder that is possible to give direct command if i know the url to retry.

Thanks and Regards,
Henry

P.S: Oh. just noticed that Creodias is one of them.

4

Dear Henry,
I would like to share information regarding availability of historical EO data on Mundi (one of the DIAS platform).
All requests (small or bigger) are currently handled by the platform. Indeed, small requests are answered directly through the web interface (https://mundiwebservices.com/account/requested). Where bigger requests can be logged through our helpdesk : https://mundiwebservices.com/contact.
Lately, Sentinel-2 L1C (from 2015 on Amazon river) and Sentinel-3 (from 2016 around Cancún, México) products were made ONLINE for users’ needs.
Let us know if this can be useful or if you need further information.

BR,
Vincent