Comments (7)
Running a mini-test using 2 gamma runs for energy estimation & 2 gamma runs + 2 proton runs for classification, I managed to get to the end.
Of course, I got warnings related to the small statistics, but I didn't get your same error.
I guess the energy model was produced without any problem.
As far as I see there are three places in which something could go wrong:
- the use of the merge script,
- the commands launched,
- the configuration file.
I propose to go step-by-step into each of these.
from protopipe.
I guess the energy model was produced without any problem.
yess I had no problems and in the merging, both the files (gamma and proton) seem fine. All the columns have values, except for the ones labeled err_est_pos
and err_est_dir
, that are nan
, but I don't think this is the problem.
I launched the script with the following command:
python adonini/protopipe/protopipe/scripts/build_model.py --config_file adonini/proto_output/configs/protopipe/classifier.yaml --tail
Instead for the dl1_write I used:
python /adonini/protopipe/protopipe/scripts/write_dl1.py
--config_file adonini/proto_output/configs/protopipe/analysis.yaml
-o adonini/proto_output/data/DL1/for_classification/${output_name}.h5
-i adonini/proto_output/data/DL0/for_classification/ -f "$filename" --tail
--estimate_energy True --regressor_dir adonini/proto_output/estimators/energy_regressor
Regarding the configuration file I only add my directory and I left the parameters you had.
from protopipe.
I found the problem: the value of the parameter offset
in divergent data. This parameter is calculated as
offset = angular_separation(run_array_direction[0],run_array_direction[1], reco_result.az, reco_result.alt)
but in divergent mode there's no "array_direction", and by default is set at 90deg. Thus the value I obtained for the offset
parameter are around 20deg, since in the MCs the source is simulated at Zd 20deg.
In the configuration file classifier.yaml
the cut for this parameter is set at 0.1 and 0.5, so in my case no data can pass these cuts.
from protopipe.
Hi Alice,
Can you confirm that this is fixed in your analysis by just modifying the model building configuration (namely, just the appropriate YAML file)?
If this is true, this is not really a bug and we can just add an appropriate section in the docs to let people know.
from protopipe.
Hi, yes it's not a bug it's just the missing of an array_direction
in divergent mode. Actually modifying the YAML file is not a good idea because you have problem later in the performance calculation. For now I hard coded the value of the array pointing in the code, but with simtel is possible to define an array_direction
in case of different telescope pointing, so with new simulations there should be no problem. Anyway regarding this we opened PR #38
from protopipe.
Perfetct! This helps to clear thing up.
So (I add to the discussion also @vuillaut, which opened said PR) if I understand correctly:
- the PR should solve, even if indirectly, this issue
- it is anyway needed for the new simulations (I assume you mean simulations dedicated to divergent pointing - if it is not already true, should the same PR take into account this?)
from protopipe.
FINAL UPDATE
This is now solved by simtel for newer divergent simulations.
from protopipe.
Related Issues (20)
- Add image extraction status
- Add tables attributes
- Merge DIRAC grid interface
- Containerization
- How to handle performance showcasing in the future HOT 1
- Refactoring of the TRAINING stage HOT 3
- Add analysis book template
- [BUG] Notebooks should try to import uproot only if ROOT-based data is required
- Add a benchmarking notebook template
- Should not transform the Geometry for each event (slow) HOT 1
- Number of recorded SST telescopes is miscalculated or missing
- Remove dangerous fall-back to MARS cuts if there is a typo in the config file HOT 2
- Some broken benchmarks HOT 1
- Update integration test pipeline with (diffuse) Prod5b data from CTAN and CTAS
- Native parallelization
- Make conda package
- PyPI upload doesn't work anymore HOT 2
- Dropdown menu to switch between docs versions doesn't work HOT 2
- Automate documentation of CLI scripts
- Should we archive this repository? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protopipe.