Comments (4)
I looked into this in some more detail. In principle, there is a tool that nicely integrates with pytest and coverage: nbval---it checks the output cell by cell, and is also sensitive to different orders (e.g. for dictionaries in py2 vs. py3) and line breaks in our !ls
statements. This can in principle be fixed by some regular expressions. Here is an example of the doc_sanitize.cfg
I created while playing around with it:
[JupyROOT version]
regex: JupyROOT\s\d.\S*
replace: JupyROOT version
[file1]
regex: additional\_figure\_\d*\.yaml\s*
replace: file
[file2]
regex: signalEffVsMass\.png\s*
replace: file
[file3]
regex: submission\.yaml\s*
replace: file
[file4]
regex: thumb\_\s*
replace: file
[file5]
regex: Figure\_\d*-\D\.png\s*
replace: file
[file6]
regex: figure\_\d\D\.yaml\s*
replace: file
Example command for testing:
py.test --nbval examples/correlation.ipynb --sanitize-with doc_sanitize.cfg --current-env
The other option, already mentioned above, is papermill, maybe together with scrapbook. Then one could write tests such as the pyhf notebook tests, and basically just make sure that the notebook still runs and in addition e.g. just check for the output files created or even just run something like that:
papermill correlation.ipynb test_papermill/correlation.ipynb --kernel python3
@AndreasAlbert What's your take on that?
from hepdata_lib.
/cc @matthewfeickert as he will have opinions here.
from hepdata_lib.
Hi @clelange, @AndreasAlbert. I would recommend papermill as @kratsg mentioned. We use it very successfully for pyhf with pytest, and I also use it to similarly validate my own work.
As you've already mentioned, scrapbook works as the compliment to papermill if you need feedback or to check notebook output.
Let me know if you have any questions.
from hepdata_lib.
Thanks for the feedback, @matthewfeickert and @kratsg - we'll probably go with scrapbook and papermill first and maybe later see whether we use nbval.
from hepdata_lib.
Related Issues (20)
- Make directory cleanup more robust HOT 2
- CMSSW setup instructions broken for CMSSW_12_X_Y and later
- Add docs for `add_additional_resource` and support `type` argument HOT 1
- Support reading arrays in C files using cfilereader
- Test all supported Python versions
- Change Zenodo record added to Submission to Concept DOI
- Drop Python 2 support HOT 3
- Yank broken Python 2 releases from PyPI with Python 3 only hepdata-validator dependencies HOT 3
- Current SWAN defaults are incompatible HOT 3
- Update readthedocs config
- Reporting a vulnerability HOT 1
- Add ability to read in YODA files HOT 10
- Pip(3) version is outdated HOT 2
- Upgrade Pylint to allow use with Python 3.11
- Add ability to specify license information HOT 2
- Allow dummy uncertainties for variable member with dummy (string) value HOT 4
- Upgrade to codecov/codecov-action@v4
- Possible workspace.json file submission + symerrors submission HOT 3
- Output with strings instead of floats HOT 3
- Support inhomogeneous uncertainties HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hepdata_lib.