Incompatibility between the config file and the way data are treated

[cta-observatory]

Incompatibility between the config file and the way data are treated

[HealthyPear]

Further information:

• this happens because the file is a La Palma-prod3b file containing also Flashcam
• using files such as this will crash also in the latest development version
• this bug is complementary to issue #23

[HealthyPear]

Update:

The cam_id_list key in the analysis.yml configuration file is used only to call the cam_ids for the estimators (one model per camera type). This means that it is completely useless in the rest of the code. Regarding --cam_ids, I am afraid that it is totally useless (I can't find it used anywhere in write_dl1.py).

What happens is that, when an ImageCleaner object is initialized, it checks which are the cameras listed in the image cleaning part of the configuration file. Since we never used Flashcam, the script will eventually crash if it doesn't find a camera that is in the simtel file.

Proposed solution:

There are a couple of nice built-in functions in ctapipe to manipulate subarrays, which have never been used probably because this part of protopipe was coded before.

For example, given a generic Paranal simtel file,

# Select all telescopes
In [22]: subarray
Out[22]: SubarrayDescription(name='MonteCarloArray', num_tels=567)
# Get the tel_ids of the LST subarray
In [15]: subarray.get_tel_ids_for_type("LST_LST_LSTCam")
Out[15]: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
# Build a custom array
In [25]: custom_array = subarray.select_subarray("custom_array", [1, 35, 90])
# Get only the cameras really involved
In [26]: [camera.cam_id for camera in custom_array.camera_types]
Out[26]: ['FlashCam', 'NectarCam', 'LSTCam']

Given PR #29 we can:

• implement the tools above in protopipe.pipeline.utils, thus making the cam_id_list key in the config file obsolete and give the list of cameras also to ImageCleaner so only the necessary objects get initialized,
• retain the camera radius part, which is necessary for the moment,
• add the missing cameras to the image cleaning section of analysis.yaml (removing the dl1_test.yaml), giving them dummy values

This should make all cases runnable, but we continue to work only on LSTCam + NectarCam for the moment. If any new developer wants to test other cameras will be more than welcomed by updated documentation (see issues #23, #27).