Problems with MAJA

[mihail.p]

Hello everyone,

I have installed Sen4Cap 2.0 and I am trying to make it operational, but I am running into some difficulties, and I decided that it can’t hurt to ask for help.

I have tried creating a few separate sites that are less than 18 months old, for the tile of 29SND (Portugal). The period I am trying out is usually between 1 and 3 months, but I am encountering some difficulties in understanding why the system is failing to work operationally.

The products are being downloaded up to a certain point, but there was a point where SciHub was mentioning that there were too many requests). I found the solution to this problem on the forum: Problem with downloading products - #12 by cudroiu

Nevertheless, what’s more important now is that MAJA (demmaccs) is always entering failed state, but I am unsure how to debug where this problem stems from. I did put the right privileges for sen2agri-services on /opt/maja, and checking them it looks as follow:

[root@my_system maja]# ls -ld
drwxr-xr-x 3 sen2agri-service root 4096 Mar 3 19:14 .

Then I checked that I have correctly installed MAJA
[root@my_system bin]# bash maja --version
2021-03-18T13:06:08.169027 my_system maja-processing-3.3.2 3.3 [000000026868] [I] MAJA Software Version : 3.3.2
OTB Version : 6.0.0
2021-03-18T13:06:08.169431 my_system maja-processing-3.3.2 3.3 [000000026868] [W] Application handler initialization: Version message

I use the usual commands of restarting services, looking at systemctl -f, and journalctl as well. These are some logs I am now getting from the command:
sudo journalctl -fu sen2agri-demmaccs --since “2 hours ago”

Mar 18 13:07:00 my_system systemd[1]: Started Runs MACCS on L1C products.
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.216376:[27349]:MainThread: Successful db fetch
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.221461:[27349]:MainThread: Successful db fetch
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.232051:[27349]:MainThread: Successful db fetch
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.235896:[27349]:MainThread: Successful db fetch
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.238126:[27349]:MainThread: Successful db fetch
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) : product 90 assigned to <worker 0>
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: #################### Tile & Site Info ####################
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: site_id = 16
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: satellite_id = 1
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: orbit_id = 37
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: tile_id = 29SND
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: downloader_history_id = 90
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: path = /mnt/archive/dwn_def/s2/default/fff/S2A_MSIL1C_20200608T112121_N0209_R037_T29SND_20200608T115350.SAFE
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: previous_l2a_path = None
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: site_short_name = fff
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: site_output_path = /mnt/archive/maccs_def/fff/l2a/
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.238413:[27349]:<worker 0>: 0: Starting extract_from_archive_if_needed for tile 29SND
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.238512:[27349]:<worker 0>: This wasn’t an archive, so continue as is.
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.238548:[27349]:<worker 0>: 0: Ended extract_from_archive_if_needed for tile 29SND
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: (launcher info) <worker 0>: Successful pre-processing = True
Mar 18 13:07:00 my_system l2a_launcher.py[27349]: 2021-03-18 13:07:00.239924:[27349]:<worker 0>: Successful pre-processing = True
Mar 18 13:07:00 my_system systemd[1]: sen2agri-demmaccs.service: main process exited, code=exited, status=1/FAILURE
Mar 18 13:07:00 my_system systemd[1]: Unit sen2agri-demmaccs.service entered failed state.
Mar 18 13:07:00 my_system systemd[1]: sen2agri-demmaccs.service failed.

Looking at this thread:

Demmaccs_launcher fail to load config

Hello, I installed Sen4Cap from 20200707_Sen4CAPDistribution_1.2.zip file. I started to create a small site to check everything is working fine. S2 L1C data has been successfully downloaded into /mnt/archive/dwn_def/s2/default/{my_site}. Yet atmospheric correction is not processed. I ran the following command from reading other topics in this forum: sudo journalctl -fu sen2agri-demmaccs --since "10 hours ago" And get the following repeated messages: Oct 27 17:22:01 va-esa-01-eu systemd[1…

and the table config in the database:

psql -U admin sen4cap -c “select * from config where key like ‘%demmaccs%’”

I noticed that this returns no rows corresponding to demmaccs. Could it be there’s an error in the installation/configuration/integration of MAJA with Sen4CAP?

Thanks for any help provided,

Greetings,

[cudroiu]

Hello,

The MAJA version that you installed is 3.3.2 while you should install 3.2.2 as the gipp files we provide are for this version (and not compatible with 3.3.2).
Steps to be made:

• install 3.2.2
• make sure you change the rights for /opt/maja cf. documentation
• update the value in the config table for key processor.l2a.maja.launcher to be /opt/maja/3.2.2/bin/maja

Please note that in version 2.0 the key demmaccs.maccs-launcher is now called processor.l2a.maja.launcher. Also, the full list of keys chaged in 2.0 is:
demmaccs.maccs-launcher => processor.l2a.maja.launcher
demmaccs.srtm-path => processor.l2a.srtm-path
demmaccs.swbd-path => processor.l2a.swbd-path
demmaccs.working-dir => processor.l2a.working-dir
demmaccs.output-path => processor.l2a.optical.output-path
demmaccs.gips-path => processor.l2a.maja.gipp-path
demmaccs.compress-tiffs => processor.l2a.optical.compress-tiffs
demmaccs.cog-tiffs => processor.l2a.optical.cog-tiffs
demmaccs.remove-sre => processor.l2a.maja.remove-sre
demmaccs.remove-fre => processor.l2a.maja.remove-fre

Hope this helps.

Best regards,
Cosmin

[mihail.p]

That was a very straightforward answer, thank you. I’m not sure how I missed that the version was 3.2.2 and not 3.3.2. I have managed to get MAJA running, but somehow now, sen2agri-services does not start, and subsequently both sources, and sites do not show any data on the web app.

When I do restart of sen2agri-services, a log appears looking like this:
● sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2021-03-19 13:20:20 UTC; 6min ago
Main PID: 9036 (start.sh)
CGroup: /system.slice/sen2agri-services.service
├─9036 /bin/bash /usr/share/sen2agri/sen2agri-services/bin/start.sh
└─9042 java -cp …/modules/:…/lib/:…/services/:…/plugins/ org.esa.sen4cap.ServicesStartup

Mar 19 13:20:25 my_system start.sh[9036]: at sun.nio.ch.Net.bind(Net.java:453)
Mar 19 13:20:25 my_system start.sh[9036]: at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
Mar 19 13:20:25 my_system start.sh[9036]: at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
Mar 19 13:20:25 my_system start.sh[9036]: at org.apache.tomcat.util.net.NioEndpoint.initServerSocket(NioEndpoint.java:230)
Mar 19 13:20:25 my_system start.sh[9036]: at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:213)
Mar 19 13:20:25 my_system start.sh[9036]: at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1124)
Mar 19 13:20:25 my_system start.sh[9036]: at org.apache.tomcat.util.net.AbstractEndpoint.start(AbstractEndpoint.java:1210)
Mar 19 13:20:25 my_system start.sh[9036]: at org.apache.coyote.AbstractProtocol.start(AbstractProtocol.java:586)
Mar 19 13:20:25 my_system start.sh[9036]: at org.apache.catalina.connector.Connector.startInternal(Connector.java:1005)

Nevertheless, when I run start.sh independently/manually, it actually runs as normal and it starts giving me the usual log of Sen4CAP, and then I’m able to see all needed data on the webpage. But the sen2agri-services itself is stuck at the aforementioned error up there. Is this some problem with permissions?

Greetings,

[cudroiu]

Dear Mihail,

My assumption is that you already have an instance of the sen2agri-services already started (one that you started manually?).
The one that is run as service cannot start because the port is already occupied by the manual launched one.
Also, please note that you should not start manually the services from another user than sen2agri-service otherwise you get write errors when they will be started as sen2agri-service.

Best regards,
Cosmin