cea-metrocarac/fitspy

Generic tool dedicated to fit spectra in python

Fitspy is a generic tool dedicated to fit spectra in python with a GUI that aims to be as simple and intuitive to use as possible.

Processed spectra may be independent of each other or may result from 2D-maps acquisitions.

Example of fitspy 2D-map frame interacting with the main GUI.

The predefined peak models considered in Fitspy are Gaussian, Lorentzian, Asymetric Gaussian, Asymetric Lorentzian and Pseudovoigt.

A constant, linear, parabolic or exponential background model can also be added in the fitting.

In both cases, user-defined models can be added.

Fitspy main features:

• Fitspy uses the lmfit library to fit the spectra
• The fit processing can be multi-threaded
• Bounds and constraints can be set on each peaks models parameter
• From an automatic noise level estimation, according to the local noise, peak models can be automatically deactivated
• Fitspy also includes automatic outlier detection to be excluded during the fitting process

All actions allowed with the GUI can be executed in script mode (see examples here). These actions (like baseline and peaks definition, parameters constraints, ...) can be saved in a Fitspy model and replayed as-is or applied to other new spectra datasets.

Installation

pip install fitspy

(See the documentation for more details)

Tests and examples execution

pip install pytest
git clone https://github.com/CEA-MetroCarac/fitspy.git
cd fitspy
pytest
python examples/ex_gui_auto_decomposition.py
python examples/ex_.......

(See the documentation for more details)

Quick start

Launch the application:

fitspy

Then, from the top to the bottom of the right panel:

• Select file(s)
• (Optional) Define the X-range
• Define the baseline to subtract (left or right click on the figure to add or delete (resp.) a baseline point)
• (Optional) Normalize the spectrum/spectra
• Click on the Fitting panel to activate it
• Select Peak model and add peaks (left or right click on the figure to add or delete (resp.) a peak)
• (Optional) Add a background (BKG model) to be fitted
• (Optional) Use Parameters to set bounds and constraints
• Fit the selected spectrum/spectra
• (Optional) Save the parameters in .csv format
• (Optional) Save the Model in a .json file (to be replayed later)

(See the documentation for more details)

Acknowledgements

This work, carried out on the CEA - Platform for Nanocharacterisation (PFNC), was supported by the “Recherche Technologique de Base” program of the French National Research Agency (ANR).

Warm thanks to the JOSS reviewers (@maurov and @FCMeng) and editor (@phibeck) for their contributions to enhancing Fitspy.

Citations

In case you use the results of this code in an article, please cite:

• Quéméré P., (2024). Fitspy: A python package for spectral decomposition. Journal of Open Source Software. doi: 10.21105/joss.05868

• Newville M., (2014). LMFIT: Non-Linear Least-Square Minimization and Curve-Fitting for Python. Zenodo. doi: 10.5281/zenodo.11813.

fitspy's Issues

Multiprocessing fails, altough fitspy works in sequential.

Fitpsy works when one CPU is selected but fails when multiprocessing is chosen.

Test issue

@patquem test

Bug related to topview Parameters

Steps to reproduce the bug:

- Load a fitted model which compose of some peak_models (JSON file)
- Apply the loaded model to a spectrum.
- Open "Parameter" and delete one or more 'peak_model'.
- Back to the GUI and apply the 'loaded model' again to the spectrum -> BUG 

==> The parameters do not show the peak parameters correctly. This is because the length of the 'peak_models' and 'peak_labels' lists is now different.

documentation missing how to run the tests and examples

@patquem

I am evaluating fitspy (openjournals/joss-reviews#5868) but I cannot run the tests and examples. The documentation should be updated with clear instructions and possibly a "quickstart tutorial". Please, could it be possible to push the documentation to a static web site? For example, GitHub Pages or Read the docs

Tested on Ubuntu WSL (Windows Subsystem for Linux), python 3.9 in a fresh conda environment.

Fitspy installation

pip install fitspy -> OK

Tests

cd
git clone https://github.com/CEA-MetroCarac/fitspy.git
cd fitspy/tests
pip install pytest

All tests fail because the examples are not found. Error trace below:

(fitspy) mauro@LAPCNRSMR:~/devel/fitspy/tests$ pytest
=============================================== test session starts ================================================
platform linux -- Python 3.9.0, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/mauro/devel/fitspy
collected 0 items / 4 errors

====================================================== ERRORS ======================================================
____________________________________ ERROR collecting tests/test_gui_2d_maps.py ____________________________________
ImportError while importing test module '/home/mauro/devel/fitspy/tests/test_gui_2d_maps.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../local/mambaforge/envs/fitspy/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test_gui_2d_maps.py:4: in <module>
    from examples.ex3_gui_2d_maps import gui_2d_maps
E   ModuleNotFoundError: No module named 'examples'
________________________ ERROR collecting tests/test_gui_apply_model_to_all_new_spectra.py _________________________
ImportError while importing test module '/home/mauro/devel/fitspy/tests/test_gui_apply_model_to_all_new_spectra.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../local/mambaforge/envs/fitspy/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test_gui_apply_model_to_all_new_spectra.py:4: in <module>
    from examples.ex2_gui_apply_model_to_all_new_spectra import \
E   ModuleNotFoundError: No module named 'examples'
______________________________ ERROR collecting tests/test_gui_auto_decomposition.py _______________________________
ImportError while importing test module '/home/mauro/devel/fitspy/tests/test_gui_auto_decomposition.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../local/mambaforge/envs/fitspy/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test_gui_auto_decomposition.py:4: in <module>
    from examples.ex0_gui_auto_decomposition import gui_auto_decomposition
E   ModuleNotFoundError: No module named 'examples'
_____________________________ ERROR collecting tests/test_inline_auto_decomposition.py _____________________________
ImportError while importing test module '/home/mauro/devel/fitspy/tests/test_inline_auto_decomposition.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../local/mambaforge/envs/fitspy/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test_inline_auto_decomposition.py:5: in <module>
    from examples.ex4_inline_auto_decomposition import inline_auto_decomposition
E   ModuleNotFoundError: No module named 'examples'
============================================= short test summary info ==============================================
ERROR test_gui_2d_maps.py
ERROR test_gui_apply_model_to_all_new_spectra.py
ERROR test_gui_auto_decomposition.py
ERROR test_inline_auto_decomposition.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================ 4 errors in 0.13s =================================================

Examples

Example 0 -> fails with error:

(fitspy) mauro@LAPCNRSMR:~/devel/fitspy/examples$ python ex0_gui_auto_decomposition.py
/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/examples/data/spectra_1 is not an existing directory
Traceback (most recent call last):
  File "/home/mauro/devel/fitspy/examples/ex0_gui_auto_decomposition.py", line 34, in <module>
    gui_auto_decomposition()
  File "/home/mauro/devel/fitspy/examples/ex0_gui_auto_decomposition.py", line 22, in gui_auto_decomposition
    appli.auto_eval_all()
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/fitspy/app/callbacks.py", line 739, in auto_eval_all
    current_fname = self.current_spectrum.fname
AttributeError: 'NoneType' object has no attribute 'fname'

Example 1 -> only empty GUI appears:

(fitspy) mauro@LAPCNRSMR:~/devel/fitspy/examples$ python ex1_gui_reload_model.py

Example 2 -> fails, data not found

(fitspy) mauro@LAPCNRSMR:~/devel/fitspy/examples$ python ex2_gui_apply_model_to_all_new_spectra.py
/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/examples/data/spectra_2 is not an existing directory

Example 3 -> fails, data not found

(fitspy) mauro@LAPCNRSMR:~/devel/fitspy/examples$ python ex3_gui_2d_maps.py
Traceback (most recent call last):
  File "/home/mauro/devel/fitspy/examples/ex3_gui_2d_maps.py", line 36, in <module>
    gui_2d_maps()
  File "/home/mauro/devel/fitspy/examples/ex3_gui_2d_maps.py", line 19, in gui_2d_maps
    appli.add_items(fnames=[str_map, unstr_map])
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/fitspy/app/callbacks.py", line 816, in add_items
    spectrum.load_profile(fname)
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/fitspy/spectra.py", line 209, in load_profile
    dfr = pd.read_csv(self.fname, sep=r'\s+|\t|,|;| ', engine='python',
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 948, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 611, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1448, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1705, in _make_engine
    self.handles = get_handle(
  File "/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/pandas/io/common.py", line 863, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/home/mauro/local/mambaforge/envs/fitspy/lib/python3.9/site-packages/examples/data/2D_maps/ordered_map.txt'

Example 4 -> fails

(fitspy) mauro@LAPCNRSMR:~/devel/fitspy/examples$ python ex4_inline_auto_decomposition.py
Traceback (most recent call last):
  File "/home/mauro/devel/fitspy/examples/ex4_inline_auto_decomposition.py", line 64, in <module>
    inline_auto_decomposition(show_plots=True)
  File "/home/mauro/devel/fitspy/examples/ex4_inline_auto_decomposition.py", line 32, in inline_auto_decomposition
    print(f"\n\nfilename: {fname}")
UnboundLocalError: local variable 'fname' referenced before assignment

Thanks for your help

Ctrl + C not working

Ctrl + C to copy fig to clipoard : not working

documentation and examples improvements

As follow-up to my comments in JOSS review I propose to improve the documentation and examples by taking into account the following points (no particular order/priority and the list is not exhaustive nor complete):

• Baseline: how to fit the baseline only?
• Baseline/GUI: the "Auto" button adds always a linear baseline, even if the polynomial is selected
• What is the difference between Baseline and "BKG model" in the Peaks section? To me, the baseline is the background model and one should be able to fit it before start adding the peak(s) models to the fit.
• GUI: would be nice to have the possibility to select multiple X-range regions to exclude/include temporary in the fit.
• The baseline should be clearly visible in the fits. For example, the left plot of ex4_inline_auto_decomposition.py should show the baseline.
• The ASCII input format of the 2D data should be clearly specified. Just pointing to the raw data is not enough and very annoying to dig into it.
• In ex3_gui_2d_maps.py when clicking on the images the data are shown in the main panel, but not the fits.
• Why normalization to 100? Usually one normalize to 1.

Please, no need to reply point-by-point here, as these are ideas only (not mandatory improvements). If you want to take them into account, I propose to commit your changes to the code/documentation/examples directly into the repository master branch.