Sen4CAP usage for historical analysis

General Discussions
1

Dear all,

I’m very new to Sen4CAP and I’m trying to figure out how to use this powerful set of processors for “historical” analysis. I mean: I’m interested in having L3A, L4A, L4B, and L4C products for a season in the past and not for the coming months.

  1. Should I use Sen4CAP in automatic mode? If not, what should I do? Of course, I have to set up the data sources. Then I suppose I have to create one or more sites: which processors should I have to select for them?

  2. I have some small areas: should I create a unique site or should I treat each of them as a separate site? In the latter case, what happens while downloading a source data intersecting more than a site? Will be the source data be downloaded as many times as the number of the intersected sites, or just once?

  3. Can I have an estimation of the volume of data to be downloaded before starting the download? Is there a way to monitor which source data have been downloaded and which not yet? Is there a way to automatically delete raw data (Sentinel-1 SLC, Sentinel-2 L1C, etc.) after L2A product generation?

  4. Are source data downloaded in chronological order (from older to newest)? And what happens with archived Sentinel data?

Thank you.
Best regards

2

Hello,

Please find below answers to your questions:

  1. To monitor a season in the past, we advise not to activate any processor in the automatic mode for the advanced processors. If you use the automatic mode, because the products dates are in the past, the system will try to launch them directly before the S1 and S2 preprocessing is ready. So we advise to wait that the S1 adn S2 preprocessing is ready before launching the advanced processors via Dashboard or Custom Jobs. Attention: if you don’t use a local root for the low-level S1 (SLC) and S2 (L1C or already preprocessed with Sen2Cor) products, the system will not be able to download these data from SciHub platfrom because they provide only a one year active archive on the platform.

  2. I don’t know about the downloading but I think that at least the preprocessing will be done for each site separately, even if some S1 and S2 data cross different sites. @cudroiu can you confirm? So I would advise to create one big site. Take also into account that the S1 low-level products are often very large.

  3. To monitor the download and preprocessing of the data:

  • monitoring tab of the GUI:
    image
    image1113×284 60.6 KB
  • in the console:
    -> connect to the system database:
    psql -U admin sen4cap
    -> check site id:
    select id,name from site;
    -> check S2 status:
    select status_id,count() from downloader_history where satellite_id=1 and site_id=1 group by status_id;
    -> check S1 status:
    select status_id,count(    ) from downloader_history where satellite_id=3 and site_id=1 group by status_id;
    -> status_id meaning:
    image
  • automatic way to delete the raw data: at my knowledge, no. @cudroiu can you confirm?
  1. Yes. And see answer 1 for the Sentinel archive.

Best regards,

Philippe

3

Dear Philippe,

thank you for your illuminating answer.

  1. In your opinion, is using a DIAS the best solution for monitoring a season in the past? In this case, may I “force” Sen4CAP to use directly Sentinel-2 L2A products instead of preprocessing L1C ones? If so, are they comparable with L2A products possibly generated by MAJA through Sen4CAP?

  2. I know S-1 low-level products are often very large. That’s why…

  3. … I was asking about having an estimation of the volume of data to be downloaded (in terms of Gigabytes). It would be better if I had a full list of products, in order to evaluate their usefulness: this would be great especially for Sentinel-1 products. But it seems that’s not possible, right?

  4. So if I ask Sen4CAP to download “offline” Sentinel products, I guess that not even the “online” products will be downloaded because they would be downloaded after the “offline” ones, right?

Kind regards

4

Dear laterza,

I will anwser those questions.
1.If you want to do historical analysis, I would advice you to use a DIAS since the download from higher than a year from the sci-hub may not be easy (very slow and not efficient).
I do not remember if the L2A from Sen2Cor are avaiblable in the DIAS and/or readable from Sen4CAP. If they are actually available, it cost less processing which is nice but the quality is less good from the MAJA processing.
2-3. yes, indeed. It is not possible to evaluated their usefulness. But using a DIAS allows to not download the images and reduce the volume of storage needed.
4. I think the online products will be still downloaded before the ‘offline’ one. To be Check !

I hope it helps and do not come too late.
Kind regards,

Diane

5

Dear Diane,

thank you, it helps a lot and it doesn’t come too late: I hope this forum will be the best companion in discovering Sen4CAP.

Kind Regards