I’m struggeling of adding a first site after installation. The shapefile is recognized and uploaded but then can not be read successfully in the database. Different browsers all show the following error msg:
Anybody encountering the same issue? I tried several shapefiles. The are all uploaded in the /mnt/upload directory. But the site is not created…
thanks for your reply! I tried many different shapefiles already. For projection I need to use EPSG:4326?
Unfortunatley I think I have still a problem with my /mnt/archive. This is a mounted nfs-share added as symlink with all permissions set but something is still wrong because in my archive folders the directories orchestrator_temp, demmaccs_tmp and the one for each site have not been created during installation.
I have almost the same problem, I have this error message when I click on “add site” “FAILED: Failed to upload the file you selected! Missing shape file due to a problem with your selected file!”. Even with the shapefiles I already have imported on my previous sites.
Thanks for your help
Maxence
hm interestingly the file is uploaded and unpacked safley in /mnt/upload but in the next step I guess it should be moved to /mnt/archive/upload but this obviously doesn’t work. Does anyone else encountered this issue so far?
about the projection for the site extent shp, it can be any projection (recognized by gdal/ogr);
you have to add the files (.shp, .dbf, .prj and .shx) directly to the zip archive -> you should not zip the folder where these files are located;
the site extent shp is uploaded in /mnt/upload/ but not in /mnt/archive/upload/ -> all the other files that you are importing in the system (parcels, LUT, config files and practice tables) are imported there, except the site extent shp.
To know a bit more about your problem, you can look at the journalctl logs. You can run the following command just after having tried again to import the shp: journalctl -e (to look at the last logs) or journactl -e | grep Error (to look for possible error). Can you spot anything?
ok I checked the journal log and there seems to be a problem with my httpd service:
setsebool -P httpd_use_nfs 1
***** Plugin catchall (11.6 confidence) suggests **************************
If you believe that httpd should be allowed unlink access on the AT.zip file by default.
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'httpd' --raw | audit2allow -M my-httpd
# semodule -i my-httpd.pp
Sep 14 13:20:38 uba-s2.novalocal dbus[1160]: [system] Activating service name=‘org.fedoraproject.Setroubleshootd’ (using servicehelper)
Sep 14 13:20:38 uba-s2.novalocal dbus[1160]: [system] Successfully activated service ‘org.fedoraproject.Setroubleshootd’
Sep 14 13:20:39 uba-s2.novalocal setroubleshoot[5274]: SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket po
Sep 14 13:20:39 uba-s2.novalocal python[5274]: SELinux is preventing /usr/sbin/httpd from name_connect access on the tcp_socket port 8080.
***** Plugin catchall_boolean (47.5 confidence) suggests ******************
If you want to allow httpd to can network connect
Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.
Do
setsebool -P httpd_can_network_connect 1
***** Plugin catchall_boolean (47.5 confidence) suggests ******************
If you want to allow httpd to can network relay
Then you must tell SELinux about this by enabling the 'httpd_can_network_relay' boolean.
Do
setsebool -P httpd_can_network_relay 1
***** Plugin catchall (6.38 confidence) suggests **************************
If you believe that httpd should be allowed name_connect access on the port 8080 tcp_socket
Then you should report this as a bug.
You can generate a local policy module to allow this access.
Do
allow this access for now by executing:
# ausearch -c 'httpd' --raw | audit2allow -M my-httpd
# semodule -i my-httpd.pp
Still struggeling with the upload. My log identifies a crash of python:
Sep 15 08:20:01 uba-s2.novalocal systemd[1]: Removed slice User Slice of root.
Sep 15 08:20:01 uba-s2.novalocal demmaccs_launcher.py[7197]: 2020-09-15 08:20:01.372829:[7197]:Could not load the config from database
Sep 15 08:20:01 uba-s2.novalocal systemd[1]: sen2agri-demmaccs.service: main process exited, code=exited, status=255/n/a
Sep 15 08:20:01 uba-s2.novalocal systemd[1]: Unit sen2agri-demmaccs.service entered failed state.
Sep 15 08:20:01 uba-s2.novalocal systemd[1]: sen2agri-demmaccs.service failed.
Sep 15 08:20:23 uba-s2.novalocal sen2agri-monitor-agent[1997]: No such file or directory
Sep 15 08:20:40 uba-s2.novalocal sen2agri-scheduler[1157]: Loading tasks from database …
Sep 15 08:20:40 uba-s2.novalocal sen2agri-scheduler[1157]: No tasks defined in the database yet!
Sep 15 08:21:08 uba-s2.novalocal systemd[1]: Started Runs MACCS on L1C products.
Sep 15 08:21:08 uba-s2.novalocal demmaccs_launcher.py[7237]: 2020-09-15 08:21:08.361586:[7237]:Could not load the config from database
Sep 15 08:21:08 uba-s2.novalocal systemd[1]: sen2agri-demmaccs.service: main process exited, code=exited, status=255/n/a
Sep 15 08:21:08 uba-s2.novalocal systemd[1]: Unit sen2agri-demmaccs.service entered failed state.
Sep 15 08:21:08 uba-s2.novalocal systemd[1]: sen2agri-demmaccs.service failed.
Sep 15 08:21:24 uba-s2.novalocal sen2agri-monitor-agent[1997]: No such file or directory
Sep 15 08:21:30 uba-s2.novalocal sshd[7244]: Accepted publickey for centos from XXX port 44570 ssh2: RSA SHA256:8F/81DEpGbtSXBGmun67lZyVikEi/3LkrE4Dy4wTcts
Sep 15 08:21:31 uba-s2.novalocal sshd[7248]: Accepted publickey for centos from XXX port 44590 ssh2: RSA SHA256:8F/81DEpGbtSXBGmun67lZyVikEi/3LkrE4Dy4wTcts
Sep 15 08:21:31 uba-s2.novalocal systemd-logind[1155]: New session 693 of user centos.
Sep 15 08:21:31 uba-s2.novalocal systemd[1]: Started Session 693 of user centos.
Sep 15 08:21:31 uba-s2.novalocal sshd[7244]: pam_unix(sshd:session): session opened for user centos by (uid=0)
Sep 15 08:21:31 uba-s2.novalocal systemd[1]: Started Session 694 of user centos.
Sep 15 08:21:31 uba-s2.novalocal systemd-logind[1155]: New session 694 of user centos.
Sep 15 08:21:31 uba-s2.novalocal sshd[7248]: pam_unix(sshd:session): session opened for user centos by (uid=0)
Sep 15 08:21:31 uba-s2.novalocal dbus[1160]: [system] Activating service name=‘org.freedesktop.problems’ (using servicehelper)
Sep 15 08:21:31 uba-s2.novalocal dbus[1160]: [system] Successfully activated service ‘org.freedesktop.problems’
Sep 15 08:21:40 uba-s2.novalocal sen2agri-scheduler[1157]: Loading tasks from database …
Sep 15 08:21:40 uba-s2.novalocal sen2agri-scheduler[1157]: No tasks defined in the database yet!
Sep 15 08:21:56 uba-s2.novalocal sshd[7326]: Accepted publickey for centos from XXX port 17916 ssh2: RSA SHA256:8F/81DEpGbtSXBGmun67lZyVikEi/3LkrE4Dy4wTcts
Sep 15 08:21:56 uba-s2.novalocal systemd-logind[1155]: New session 695 of user centos.
Sep 15 08:21:56 uba-s2.novalocal systemd[1]: Started Session 695 of user centos.
Sep 15 08:21:56 uba-s2.novalocal sshd[7326]: pam_unix(sshd:session): session opened for user centos by (uid=0)
Sep 15 08:22:08 uba-s2.novalocal systemd[1]: Started Runs MACCS on L1C products.
Sep 15 08:22:08 uba-s2.novalocal demmaccs_launcher.py[7342]: 2020-09-15 08:22:08.441850:[7342]:Could not load the config from database
Sep 15 08:22:08 uba-s2.novalocal systemd[1]: sen2agri-demmaccs.service: main process exited, code=exited, status=255/n/a
Sep 15 08:22:08 uba-s2.novalocal systemd[1]: Unit sen2agri-demmaccs.service entered failed state.
Sep 15 08:22:08 uba-s2.novalocal systemd[1]: sen2agri-demmaccs.service failed.
Sep 15 08:22:25 uba-s2.novalocal sen2agri-monitor-agent[1997]: No such file or directory Sep 15 08:22:27 uba-s2.novalocal abrt-hook-ccpp[7358]: Process 7357 (python2.7) of user 53402 killed by SIGABRT - dumping core Sep 15 08:22:28 uba-s2.novalocal abrt-server[7359]: Interpreter crashed, but no packaged script detected: 'python scripts/check_shp.py -b /mnt/upload/AT3/sen4cap_1601758305942/AT_MMB.s Sep 15 08:22:28 uba-s2.novalocal abrt-server[7359]: ‘post-create’ on ‘/var/spool/abrt/ccpp-2020-09-15-08:22:27-7357’ exited with 1 Sep 15 08:22:28 uba-s2.novalocal abrt-server[7359]: Deleting problem directory '/var/spool/abrt/ccpp-2020-09-15-08:22:27-7357’ Sep 15 08:22:40 uba-s2.novalocal sen2agri-scheduler[1157]: Loading tasks from database … Sep 15 08:22:40 uba-s2.novalocal sen2agri-scheduler[1157]: No tasks defined in the database yet! Sep 15 08:22:49 uba-s2.novalocal unix_chkpwd[7379]: password check failed for user (root) Sep 15 08:22:49 uba-s2.novalocal su[7375]: pam_unix(su:auth): authentication failure; logname=centos uid=1000 euid=0 tty=pts/0 ruser=centos rhost= user=root Sep 15 08:22:49 uba-s2.novalocal su[7375]: pam_succeed_if(su:auth): requirement “uid >= 1000” not met by user "root" Sep 15 08:22:51 uba-s2.novalocal su[7375]: FAILED SU (to root) centos on pts/0 Sep 15 08:22:58 uba-s2.novalocal su[7380]: (to root) centos on pts/0 Sep 15 08:22:58 uba-s2.novalocal su[7380]: pam_unix(su:session): session opened for user root by centos(uid=1000)
Now I ran the check_shp.py script and got the following in return:
@philippe, here are the logs, we don’t understand the issue
сен 15 07:10:04 sen4cap systemd[1]: Started Runs MACCS on L1C products.
сен 15 07:10:04 sen4cap demmaccs_launcher.py[11566]: 2020-09-15 07:10:04.900558:[11566]:Could not load the config from database
сен 15 07:10:04 sen4cap systemd[1]: sen2agri-demmaccs.service: main process exited, code=exited, status=255/n/a
сен 15 07:10:04 sen4cap systemd[1]: Unit sen2agri-demmaccs.service entered failed state.
сен 15 07:10:04 sen4cap systemd[1]: sen2agri-demmaccs.service failed.
сен 15 07:10:21 sen4cap kernel: type=1400 audit(1600153821.247:445): avc: denied { read } for pid=31258 comm=“httpd” name=“lesattaquestwo” dev=“dm-0” ino=118096430 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tcl
сен 15 07:10:21 sen4cap kernel: type=1400 audit(1600153821.251:446): avc: denied { getattr } for pid=31258 comm=“httpd” path="/mnt/archive/upload/lpis/lesattaquestwo/SHALAWGS.shx" dev=“dm-0” ino=22151182 scontext=system_u:system_r:httpd_t:s0 tcont
сен 15 07:10:23 sen4cap sen2agri-scheduler[644]: Loading tasks from database …
сен 15 07:10:23 sen4cap sen2agri-scheduler[644]: No tasks defined in the database yet!
сен 15 07:10:28 sen4cap sen2agri-monitor-agent[1364]: No such file or directory
сен 15 07:10:58 sen4cap kernel: type=1400 audit(1600153858.353:447): avc: denied { write } for pid=11211 comm=“httpd” name=“upload” dev=“dm-0” ino=7340033 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=d
сен 15 07:10:58 sen4cap kernel: type=1400 audit(1600153858.358:448): avc: denied { add_name } for pid=11211 comm=“httpd” name=“Test” scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=dir permissive=1
сен 15 07:10:58 sen4cap kernel: type=1400 audit(1600153858.362:449): avc: denied { create } for pid=11211 comm=“httpd” name=“Test” scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=dir permissive=1
сен 15 07:11:02 sen4cap systemd[1]: Started Runs MACCS on L1C products.
сен 15 07:11:02 sen4cap demmaccs_launcher.py[11630]: 2020-09-15 07:11:02.959236:[11630]:Could not load the config from database
сен 15 07:11:02 sen4cap systemd[1]: sen2agri-demmaccs.service: main process exited, code=exited, status=255/n/a
сен 15 07:11:02 sen4cap systemd[1]: Unit sen2agri-demmaccs.service entered failed state.
сен 15 07:11:02 sen4cap systemd[1]: sen2agri-demmaccs.service failed.
From the logs I see some “access denied” messages.
Could you check that the file /mnt/upload/AT/sen4cap_1602451417639/AT_MMB.shp really exists?
If not, could you give 777 access for /mnt/upload ?
Also, you could do a restart of the httpd service (sudo systemctl restart httpd).
If the /mnt/upload/AT/sen4cap_1602451417639/AT_MMB.shp exists, could you check that the following command under the user sen2agri-service and not under root (sudo su -l sen2agri-service) is executed ok:
python /var/www/html/scripts/check_shp.py -b /mnt/upload/AT/sen4cap_1602451417639/AT_MMB.shp
If you still obtain the python error with “Memory map” message above, you should check that the gdal bindings. Did you installed other versions of gdal on that machine or used pip to add/remove some packages that were related to gdal? Can you check the following:
Under the user sen2agri-service (sudo su -l sen2agri-service):
/usr/local/bin/gdal_translate --version
/usr/bin/gdal_translate --version
gdal-config --version
python
>> import osgeo.gdal
>> print osgeo.gdal.__version__
Following your suggestions. I think the issue is caused by gdal.
[sen2agri-service@uba-s2 ~]$ /usr/local/bin/gdal_translate --version
GDAL 2.3.2, released 2018/09/21
[sen2agri-service@uba-s2 ~]$ /usr/bin/gdal_translate --version
GDAL 1.11.4, released 2016/01/25
[sen2agri-service@uba-s2 ~]$ gdal-config --version
2.3.2
[sen2agri-service@uba-s2 ~]$ python
Python 2.7.5 (default, Aug 7 2019, 00:51:29)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import osgeo.gdal
>>> print osgeo.gdal.__version__
1.11.4
>>>
Probably I need to update the version for the sen2agri user to GDAL > 2?
I also spotted in the service.properties file the flag for using a GDAL docker container
Would this be a solution? Installing docker, pulling the geodata/gdal image and setting the flag to true?