We have created a site. Images start downloading but within a couple of seconds we get the message in the monitoring tab that a large percentage of them are marked as “Retriable”. After that nothing happens.
We have followed the guideliens appearing a different post (Transferring products into a fresh install) but again nothing happened.
Some of the steps we tried:
- Changed from dhus to apihub
- Changed to the latest apihub url (20/5/2021)
- MaxConnections for scihub are set to 1
This is our scihub.properties content
scihub.search.url = https://apihub.copernicus.eu/apihub/search
#scihub.search.url = https://scihub.copernicus.eu/dhus/search
#scihub.search.count.url = https://scihub.copernicus.eu/dhus/api/stub/products/count
scihub.search.count.url = https://apihub.copernicus.eu/apihub/search
scihub.product.url = https://apihub.copernicus.eu/apihub/odata/v1
scihub.product.backup.url = https://scihub.copernicus.eu/dhus/odata/v1
progress.interval = 10000
We get some messages in the log file in the following form
Fetching product 'S2A_MSIL1C_20200729T090601_N0209_R050_T34SFH_20200729T112307' failed: java.io.IOException: Product S2A_MSIL1C_20200729T090601_N0209_R050_T34SFH_20200729T112307 is not online
Thanks in advance,
Nikos
Dear Nikos,
This is because ESA changed few weeks ago the online products retention policy (the interval for how long the products are kept into the online archive) from 1 year to only 1 month 
, according to this:
https://scihub.copernicus.eu/twiki/do/view/SciHubUserGuide/LongTermArchive#Online_Products_Retention_Policy
Best regards,
Cosmin
Cosmin, thanks again for the reply (and all the previous ones!).
How does this affect Sen4Cap though?
I can see that
The download request of an offline product, automatically triggers the request for restoring the product back online from the LTA. Once restored (eg: online) the product(s) can be downloaded by the user.
which explains the Retriable message but what is the expected behaviour now?
Will Sen4Cap retry later? (and how much later and for how long?)
Is there a “best practice” recommended for such cases?
Dear Nikos,
Normally, the Sen4CAP services will retry the failed products from hour to hour.
If a product that is offline becomes online, it will be downloaded.
Unfortunately, the SciHub is implementing also a quota for the number of products downloaded per day so in the end, the system will no longer be able to download other products, probably until next day. See also the discussions from here on this subject:
But if you have a large site with a large season, it will take a very long time to download all the products you need.
We just hope that ESA will come back to this limitation(s) and will relax the conditions (to have online products for 1 year as before).
Otherwise, another solution will be to move with the production on a DIAS.
Best regards,
Cosmin