We have been running into an issue with the Sen4CAP L4A Crop Type processor with several clients and ourselves. We are using the latest version (1.2) and we have made sure that the LUT and declaration data is in the right format. All data is imported correctly without any errors. After we run the L4A classifier, there are no errors and expected data is created, so all seems to work fine. The confusion matrices, etc. are created and make sense. So the RF classifier does what it needs to do.
The issue is that we never get any predicted classes, the columns CT_decl, CT_pred_1, CT_conf_1, CT_pred_2 and CT_conf_2 are all empty. Initially I thought the issue was related to the size of the test data (we had two areas with about 5k and 20k parcels), but we just came from a call with a client with the exact same problem with a 200k sample size. From the Forum I also got the impression more users have this problem.
Would anybody be able to give us a helping hand here figuring out what the problem is?
We have had this same issue. As of now we have not come up with a solution, but a workaround exists:
By running the python script crop-type-wrapper.py manually as Philippe instructed here we were able to produce and access L4A results successfully. Only needed to manually join them to the declaration data set.
As this is the case, we suspect the issue lies with the final exporting step of the L4A processor.
If it’s a new install it might be a known issue, caused by a change in a library that the classification script uses. On some systems it’s crashing when computing the accuracy statistics.
If you don’t mind trying a patch, I’m attaching an updated version of the classification script (can be placed in /usr/bin): crop_type.zip (6.7 KB).
As Sakari mentioned, can you check the Parcels_all_with_predictions_%.csv and Parcels_classified_with_predictions_%.csv files? Are the predictions also empty in these files (CT_decl, CT_pred_1, CT_conf_1, CT_pred_2 and CT_conf_2)? If this is not the case, it must be a problem in the export of the shp.
Another observation: it is not a blocking point, but with small datasets, the parameters of the strategy for the selection of the calibration parcels for each crop type should be adapted. These parameters are explained in the ATBD for L4A crop type mapping 1.2 (http://esa-sen4cap.org/content/technical-documents), P;30-31. With the by default parameters (–best-s2-pix 10 --pa-min 30 --pa-train-h 4000 --pa-train-l 1333 --sample-ratio-h 0.25 --sample-ratio-l 0.75 --smote-target 1000):
Only crop types with more than 30 parcels containing at least 10 S2 pixels will be classified
For crop types with more than 4000 parcels (containing at least 10 S2 pixels), 25% of the parcels are used for the calibration
For crop types with less than 4000 but more than 1333 parcels (containing at least 10 S2 pixels), 1000 parcels will be used for the calibration (which represents between 25% to 75% of the parcels)
For crop types with less than 1333 parcels (containing at least 10 S2 pixels), 75% of the parcels are used for the calibration and then SMOTE process is applied to artificially create new parcels, and reach 1000 parcels for the calibration
These parameters can be changed in the config table, OR when launching a L4A crop type via the custom jobs tab by showing and modifying the advanced parameters, OR when launching a L4A crop type manually.
A last element: in the outputs, you can also find a Crop_types_summary_%.csv file, where you can see for each crop type, the strategy that was applied for this crop type (1, 2, 3 or 0 (not classified)).
In my case there are no files named Parcels_all_with_predictions_.csv and Parcels_classified_with_predictions_.csv in the L4A data that is produced, see screenshot
It seems that the argument --lc is not defined well in your command: by default it is defined as 1234, and can have only the values specified your message error. Did you launch the crop-type-wrapper.py manually? If yes, can you check this?
I ran crop-type-mapper.py from the command line as well, and got the same error when the crop_type.R script was attempted to be run, see below. It seems the R script does not parse the --lc command well.
I did a bit more research on the crop_type.R script myself and noticed an error occurs because of a missing InputOptRe variable, which is a table of extracted Red Edge data. For some reason this data is not extracted in the extraction phase before the R script is run.
I set this variable to 0 in the command sent to R, ran the script again and then I get results, see below. So if this OptRe issue is fixed, all should work ok.
Thanks for that. Indeed, I encoutered the same problem on another machine. I will discuss it with the team. Meanwhile, you can check in the /mnt/archive/orchestrator_temp/s4c_l4a/1688/13407-s4c-crop-type/optical/ folder if you have this file: optical-features-re.csv. If this is the case, you can move it to the /mnt/archive/orchestrator_temp/s4c_l4a/1688/13407-s4c-crop-type/features/ folder and use it in the crop_type.R script, to use also in the classification the red-edge features.
Sorry for the delay. In fact, there was a mismatch in the patch shared by Laurentiu, which integrates already some improvements that we will do in the next version of Sen4CAP (to accelerate the process). In version 1.2, you should use this patch (crop_type.zip (6.6 KB) ), because the red-edge features are calculated at the same time as the other optical features, and are included in the same csv. You can check them in the /features/optical-features.csv file (b5, b6 an b7).
Can you replace (again, sorry) the crop_type.R script, and do a test? It would be very great if you can let us know about the results.
Good news: I got a complete result with all the expected outputs when I used the script you sent.
The only thing I noticed is that I got an initial error the system could not find the ‘crop_type.R’ file, but I solved that with a dos2unix command. Apparently it was saved in a DOS environment.
I have the same Issue: no prediction in the croptype results. I did already change the crop_type.R script and restarted the sen2agri services. Still I get empty prediction columns. I might have missed some details in this thread. Are there other changes, except replacing the R-script that I need to make, to get a proper result? By the way I hope I replaced the right file at: /usr/bin/crop_type.R.
Did you run it by custom job or you run the command? What I observed is it went well when I run the script from terminal. But still get prediction column is 0 for some crop when i run from custom job.
I started the L4a via console. At the end of processing I get a warning (below) and I am missing the result crop_type.csv file. Can you make sense of that warning message? Any other Ideas?
Have a nice weekend everybody!
Warning message:
NAs introduced by coercion
`summarise()` ungrouping output (override with `.groups` argument)
Warning message:
NAs introduced by coercion
Error in `levels<-`(`*tmp*`, value = as.character(levels)) :
factor level [23] is duplicated
Calls: factor
Execution halted
Exit code: 1
tibble 3.0.3 
dplyr::filter() masks stats::filter()