I have run into an issue when executing the L4B processor (sen4cap version is 2.0 on a completely fresh machine) and was wondering whether you could give me a hand in this. We have run this processor successfully on v1.2 before.
The L4B processor successfully finishes MowingInputShpGenerator and S2MowingDetection steps, but errors out on the S1MowingDetection step with an error message:
/home/sen2agri-service/miniconda3/envs/sen4cap/lib/python3.6/site-packages/scipy/ndimage/measurements.py:668: RuntimeWarning: invalid value encountered in true_divide
return sum / numpy.asanyarray(count).astype(numpy.float64)
Traceback (most recent call last):
File “/usr/share/sen2agri/S4C_L4B_GrasslandMowing/Bin/src_s1/S1_main.py”, line 854, in
main()
File “/usr/share/sen2agri/S4C_L4B_GrasslandMowing/Bin/src_s1/S1_main.py”, line 848, in main
do_cmpl=s4cConfig.do_cmpl, test=s4cConfig.test)
File “/usr/share/sen2agri/S4C_L4B_GrasslandMowing/Bin/src_s1/S1_main.py”, line 584, in main_run
compliancy_conf_file=configFile)
File “/usr/share/sen2agri/S4C_L4B_GrasslandMowing/Bin/src_s1/S1_main.py”, line 151, in run_proc
srcNodata=invalid_data, resampling=resampling, error_threshold=error_threshold)
File “/usr/share/sen2agri/S4C_L4B_GrasslandMowing/Bin/src_s1/S1_gmd.py”, line 246, in make_vrt
data_proj = gdal_data.GetProjection()
File “/home/sen2agri-service/miniconda3/envs/sen4cap/lib/python3.6/site-packages/osgeo/gdal.py”, line 2272, in GetProjection
return _gdal.Dataset_GetProjection(self, *args)
RuntimeError: PROJ: proj_uom_get_info_from_database: unit of measure not found
From what I gather, this issue is due to GDAL working with a dataset that has a user-defined CRS.
This topic on GIS Stack Exchange mentions a very similar situation:
The proposed solution seems to be to update the GDAL library.
So my question is - have you run into this problem? If GDAL being out of date is at fault, would it be possible to update GDAL within the system and not cause conflict with other dependencies? If so, could you provide the appropriate instructions?
I have run into the same problem myself. I tried updating libgeotiff (the underlying source of the error according to the referenced github issue) but it didn’t help and I managed to break even stuff that was working before
However, when I tried running L4B processor with only a subset of S1 products I found out that the problem is not necessarily caused by faulty lib, because it is working fine for most of the S1 products. Turns out that in my case out of almost 100 input products only 2 were causing the error, so I decided to ignore the faulty ones.
It can be done by editing the file /usr/share/sen2agri/S4C_L4B_GrasslandMowing/Bin/src_s1/S1_gmd.py
Originally, line 246 looked like this
data_proj = gdal_data.GetProjection()
I have wrapped it in try-except so that the execution could continue:
System will consider the file as not having a geoprojection and discard it, but continue without crash.
The products themselves are still probably faulty which may cause issues to other processors as well. I will write an update if that happens.
Such a simple yet effective solution. I wonder how I didn’t think of this. I did the changes you proposed and now the processor works as it should. Thanks a lot!
Hi,
I have the same problem but unfortunately there are so many products with the error that i do not get any mowing detections based on only S1. I followed the link you have posted and it seems that the newest gdal update corrected the error but have you tried to update GDAL and checked if it is not causing any other problems?
Hi,
yes no problem. I am running the L4B with only S1 from the terminal. I have attached a folder with one of the coherence tif files which is flagged with an error. image1 in the folder shows the terminal output without any changes in the S1_gmd.py script. image2 shows the main error which shows upp very often when I add the changes adam.knaze suggested for the S1_gmd.py script. With the changes, the whole L4B finishes and does not terminate before. terminaloutout.txt is everything the terminal shows when i run L4B so you will find the error in that text file plenty of times. But i am getting no mowing events in the output shapefile so i am guessing it is because so many products are skipped. Ny mapp.zip (1.9 MB)
I’m not sure what’s wrong. I can reproduce that error (well, it’s a warning for me) on my computer (outside of Sen4CAP), but it works in an OSGeo Docker container. I think the processor is already running in a container, so we could upgrade. On the other hand, the code that’s crashing is not used.
But the strange part is that the whole raster is black. Are the others like that, too? You can check by running gdalinfo -stats on them.
I checked another product and its the same problem. So a coherence tif is created but it has only 0 as a value. There is also no amplitude product created and the log shows that it got skipped. I have attached another tif and the log to it. There is a lot of these no data tifs in our archive so that S1 would not really contribute much to an analysis i fear. But I think its odd that these products are not flagged as faulty since they do not contain real data. Hopefully you can find a fix for it. output_20210817.zip (658.7 KB)
this is an old thread now, but I’ve recently been having a lot of the same issues as described here - S1 processing resulting in coherence products with just 0s and amplitude product skipped. Did you manage to resolve the issue back then? Thanks for any tips.
Hi Adam,
I checked my old mails that I had with @cudroiu about this issue. He first ran a script to find all of the products and then deleted them so that they can get reprocessed. After that, the productas were fine but I could not find the reason for the faulty products in the mail conversation.
/ Bastian
