pyellips/pyelli

An open source ellipsometry analysis tool for reproducible and comprehensible building of optical models.

---

pyElli

PyElli is an open source numerical solver for spectral ellipsometry employing well-known 2x2 and 4x4 algorithms. It is intended for a broad case of problems including simple fitting of layered structures, anisotropic layers and any other light interaction with layered 1D structures. It serves as a system for the day to day ellipsometry task at hand and is easily extendable with your own dispersion models, EMAs or solvers. Our goal is to provide a reproducible and flexible tool for the needs of scientists working with spectral ellipsometry.

Features

• A multitude of models to approximate the dielectric function of your material.
• Use the vast library of materials from refractiveindex.info as reference materials.
• Build up your structure easily from materials and layers.
• Simulate reflection and transmission spectra, ellipsometric parameters and Mueller matrices.
• Utilities to quickly convert, plot and fit your measurement data.
• Powerful when necessary, editable and expandable.

Got a question?

If you have questions using pyElli please feel free to open a discussion in the Q&A or join our discord channel.

How to get it

The installers for all releases are available at the Python Package Index (PyPI).

To install run:

pip install pyElli[fitting]

This installs pyElli with the additional fitting capabilities and interactive widgets. If you don't want to have this functionality just drop the [fitting] in the end.

To increase performance of the 4x4 Solver, it is recommended to install PyTorch manually, as it is too big to include in the standard installation. Installation information can be found at the PyTorch Website. The CPU variant is sufficient, if you want to save some space.

A complete environment for pyElli is also available as a Docker Container. To pull and run it directly just execute

docker run -p 8888:8888 domna/pyelli

from your local docker install. After startup a link should appear in your console. Click it and you will be directed to a jupyter server with the latest release of pyElli available.

To install the latest development version use:

pip install "pyElli[fitting] @ git+https://github.com/PyEllips/pyElli.git"

The source code is hosted on GitHub, to manually install from source, clone the repository and run pip install -e . in the folder to install it in development mode:

git clone https://github.com/PyEllips/pyElli
cd pyElli
pip install -e ".[fitting]"

Acknowledgements

• Based on Olivier Castany's Berreman4x4
• Solver2x2 based on Steve Byrnes' tmm
• Mikhail Polyanskiy's refractiveindex.info database and Pavel Dmitriev's pyTMM for his importer script for the database

@MarJMue receives financial support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), grant No. 398143140 (FOR 2824).

pyelli's Issues

Action for automatically building and pushing docker image to docker hub

Build an action according to Publish images to docker hub.
Should work out of the box, however, I cannot access Settings in the fork.

@MarJMue : Kannst du die Settings für den Fork einstellen und da Secrets hinzufügen? (s. Encrypted secrets)

Generalise NeXus importer

Currently, the nexus import can only deal with simple psi/delta files and ignores additional measurement parameters and times.
This may be generalised in two fields:

• Additional fields such as the free parameter and time should be read out from the nxs file as well. Here it is maybe useful to use xarray instead of pandas. This would require an update of the decorators - but it is also possible to merge the simple psi/delta decorator into a multi fit decorator to just deal with xarray structures.
• The labels should just be taken from the appropriate fields in the nxs, such that the resulting structure represents the underlying file. This requires a change in the decorators, too (see #53).

Decorator parameters fail if they do not have max/min values

Fitting decorator fails if plotly is not set as backend in pandas

Create a manual plotly plot instead of using pd.DataFrame.plot()

Flipped signs in absorptive Mueller matrix elements

The m_43 and m_34 elements in the Mueller matrix are flipped. This is probably due to the n + ik sign convention. However, we should check if anisotropic elements are affected by the sign convention as well to have a valid transformation between the two conventions.

Fix non-isotropic half space

Fix the eigenvalue solving and sorting for the non-isotropic half space case.

Implement Kramers-Kronig relations

To supply a wider range of optical models it is necessary to use kkr's, because for some dispersions only the real part is analytically defined.
This relates directly to #55.

Update examples to new code

Reimplement and check the examples against the old code.
Maybe they provide to be useful for documentation and automatic tests.

Implement new interact decorator for lmfit parameters

To roughly estimate the start parameters of a model it would be helpful to have a simple decorator similar to ipywidgets @interact, which takes an lmfit Parameters object and presents sliders for each parameter.

Examples in Documentation

Ideally integrate the Jupiter Notebook Examples directly into the documentation, including plots (ideally interactively with plotly).

Müller Matrix Calculation has wrong signs

Last row of matrix elements has the inversed sign for isotropic materials.

Anisotropic samples need to be investigated to get insight into other elements.

Multi-experiment result handling

Write a data handling tool for experiments, with varying parameters e.g. layer thickness or incidence angle.

Should provide same functionality as Result class but for a list of Results.

Delete old benchmarks and copy data from test_solver4x4_scipy to test_solver4x4_expm

Performance benchmarks

• to enable regression detection
• ideally automated 😎

Calculation for non-isotropic back-halfspace is unstable

The ordering of eigenvalues of the Delta matrix fails for the case of total internal reflection and results in a singular transfer matrix.

Add explanatory texts and examples to api documentation

Each class should have a small introductory and explanatory text with an example on how to use them similar to the texts in the dispersions and solvers documentation.

The texts should be in the docstring of the respective classes.

Data export for decorators

Add functions for decorations to

• Save data to csv file
• Export it in the ipynb

Fit checkboxes for decorators

Display checkboxes beside each fitting parameter to include or exclude it in the fit.

Multi-Experiment fitting decorator

A fitting decorator similar to the mmatrix and psi_delta decorator to fit multiple experiments on the same sample simultaneously (e.g. different angles of incidence)

Image to show the relations between Material, Structure, Experiment and Result

Relates to #61

Something like this:

Add a NeXus exporter

After reading (#50) and analysing the data, the processed data should be written as an NXprocess entry to the NeXus measurement file.

General introduction docs

There should be a general introduction to the software and what its goals are. Also it there should be an easy understandable short overview and "cheatsheet".

Documentation

Create a rudimentary documentation (at least some ipython examples, which can be viewed in the wiki)

Data loader for refractiveindex.info

To supply many tabulated materials it would be helpful to have a loader for refractive index.info data.

For now we could just work with a plugin which takes the url of a csv file loading it into a tabular dispersion. Later I could imagine that we even could have some form of material search that one just has to type in the materials name.

Prevent VaryingMixtureLayer from changing Material

The inhomogeneous layer changes the fraction of the mixture material during evaluation, which should not happen.

New DispersionLaw Architecture

Rewrite the DispersionLaws into Dataclasses with some kind of plugin / factory architecture and the ability to generate standardized parameters for fitting.

Reimplement inhomogenius materials

The rewrite broke the calculation of inhomogeneous materials (liquid crystals etc.).

Design a Logo

Finishing the design of a logo/icon for pyElli.

Current State:

Colors still up for debate.

Don't use lambda as kwarg in dispersions

In some dispersions lambda is used as kwarg to indicate a central wavelength (e.g. for LorentzLambda). Since this is the keyword for python lambda functions it is not possible to actually use it as a kwarg but just as an arg. Therefore all occurrences should be renamed.

Generalize column names in fitting decorators

When data is supplied to the decorators they assume that there are two columns with the name Ψ/Δ.
This probably should be stronger generalised, w/ supplying the appropriate name in the fit function.
Additionally, it is probably better to use psi and delta instead of the greek letters.

Cleaner Module Overview

The overview of modules should be cleaner and more accessible to readers unfamiliar with the code.
Currently, it's just a display of module names.

Add residual to decorators dropdown

Interactive fitting decorator

Maybe its cool to add a fitting button and update the fitting graph at each iteration with lmfits minimizer iter_cb function. This is probably slow, but maybe its possible to handle this asynchronously.

Decorator throws error while plotting if plotly is not the pandas backend

The error originates here:

File /opt/conda/lib/python3.10/site-packages/elli/fitting/decorator_psi_delta.py:268, in FitRho.plot(self)
    265 """Plot the fit results as Psi/Delta"""
    266 fit_result = self.model(self.exp_data.index.to_numpy(), self.fitted_params)
--> 268 return go.FigureWidget(
    269     pd.concat(
    270         [
    271             self.exp_data,
    272             pd.DataFrame(
    273                 {"Ψ_fit": fit_result.psi, "Δ_fit": fit_result.delta},
    274                 index=self.exp_data.index,
    275             ),
    276         ]
    277     ).plot()
    278 )

Eventually, FigureWidget throws an error because it doesn't know what it's supposed to do with a matplotlib object.

Reformat Dispersions documentation

Reformat the documentation for the Dispersion classes to be better readable.
Currently, there are major formatting issues.

Create a non forked repository

Create a non forked repository and create an organization for the project.

Support for EMA layers

Allow the construction of an EMA layer between two materials. This should support various EMA functions, e.g. at least constant fraction, Bruggeman, Maxwell-Garnett. A variation of fraction along the z-axis should be possible, too.

Integrate a tensorflow 2x2 version

As in: https://github.com/fancompute/vtmm/blob/master/vtmm/tmm.py

Fitting class

A decorator for declaring a function as a layered optical model, which automatically initialiases the necessary parameters for lmfit and (automatically?) triggers a fit.

History wrapper for lmfit Parameters class

A wrapper to which changes in lmfits Parameters classes can be commited to build a history of fits and a convenient workflow.

Update contributing guidelines

The contributing guidelines should reflect what the project stands for (open source tools for ellipsometry) and should be more specific to the project.

Ideally, it contains also a short section on how to add optical models to pyElli, as I think this will be the first things being contributed.

Unified data storage object

Maybe explore the idea of an internal data structure object (dataframe or xarray), which gets populated with importers.
Should make #32 and #53 somewhat easier.

Action for automatic pushing to a package repository (pypi?)

After a new release a package should be automatically created and pushed to a python package repository.

TiO2 Test

Proper dependency management

Introduce better dependency management to not run in any dependency issues.
Either use poetry, pipenv or specify version in requirements.txt with pip freeze > requirements.txt or add an additional requirements.freeze.txt.

This includes offering better install options for the optional fitting part. I think we should have something like pip install pyElli[fitting].

Complete set of optical models

Have at least all models from Fujiwara implemented.

Consistent code style

Run all subfiles through a linter and an formatter, to get a consistent style.

Part of the code is already formatted with black, so it should be used everywhere.

Parameter estimation decorator for pseudo dielectric function

For unknown materials it is convenient to estimate the first parameterset from the pseudo dielectric function. Therefore, a decorator similar to the manual_parameters decorator would be useful.

Reimplement the Result get methods

Make it possible again, to get specific matrix elements by their physical name: E.g. R_ss

Add a NeXus importer

Build an importer for NeXus files to read standardised ellipsometry data and to finally integrate it as an analysis example in NOMAD.

NeXus ellipsometry extension