laurentrdc/scikit-ued

Collection of algorithms and routines for (ultrafast) electron diffraction and scattering

scikit-ued

Collection of algorithms and functions for ultrafast electron diffraction. It aims to be a fully-tested package taking advantage of Python's most recent features.

For examples, see our tutorials.

API Reference

The API Reference on readthedocs.io provides API-level documentation, as well as tutorials.

Installation

scikit-ued is available on PyPI; it can be installed with pip:

python -m pip install scikit-ued

To also install optional dependencies required to view diffraction images interactively:

python -m pip install scikit-ued[diffshow]

scikit-ued is also available on the conda-forge channel for the conda package manager:

conda config --add channels conda-forge
conda install scikit-ued

To install the latest development version from Github:

python -m pip install git+https://github.com/LaurentRDC/scikit-ued.git

After installing scikit-ued you can use it like any other Python module as skued.

Each version is tested against Python 3.7+. If you are using a different version, tests can be run using the pytest package.

Optional dependencies

For displaying diffraction images with interactive contrast using the skued.diffshow function, PyQtGraph is required.

Contributing

If you want to contribute to scikit-ued, take a look at CONTRIBUTING.md.

Related projects

Streaming operations on NumPy arrays are available in the npstreams package.

Interactive exploration of ultrafast electron diffraction data with the iris-ued package.

Crystal structure manipulation (including symmetry-determination) with the crystals package. (Included with scikit-ued)

A graphical user interface for the dual-tree complex wavelet transform baseline-removal routine is available as a separate package.

Citations

If you find this software useful, please consider citing the following publication:

L. P. René de Cotret, M. R. Otto, M. J. Stern. and B. J. Siwick, An open-source software ecosystem for the interactive exploration of ultrafast electron scattering data, Advanced Structural and Chemical Imaging 4:11 (2018) DOI: 10.1186/s40679-018-0060-y.

If you are using the baseline-removal functionality of scikit-ued, please consider citing the following publication:

L. P. René de Cotret and B. J. Siwick, A general method for baseline-removal in ultrafast electron powder diffraction data using the dual-tree complex wavelet transform, Struct. Dyn. 4 (2017) DOI: 10.1063/1.4972518.

Support / Report Issues

All support requests and issue reports should be filed on Github as an issue.

License

scikit-ued is made available under the GPLv3 License. For more details, see LICENSE.txt.

scikit-ued's Issues

autocenter() fails with some SAED pattern

Version / Platform Info

• scikit-ued Version: 2.1.7
• Happens in latest version? (Yes/No): yes
• Known Operating System(s) affected__:
  • [ x] Windows
  • Linux
  • Mac OS

Expected Behavior

autocenter() should find the pattern center

Actual Behavior

It results a wrong pattern center

Minimal Example of Issue

from skued import diffread, autocenter
import numpy as np
import matplotlib.pyplot as plt
from skimage.draw import polygon2mask

im = diffread('SAED_pattern.dm3')

fig, ax1 = plt.subplots(figsize = (4.5, 4.5))
ax1.imshow(im, cmap='gray', vmin=0, vmax=1000)

########## mask the beam stopper ########
polygon=np.array([[ 821.06421356, 1232.58513709],
[1128.41197691, 1368.52741703],
[1737.1969697 , 9.1046176 ],
[1347.1017316 , 9.1046176 ]])
mask = polygon2mask(im.shape, polygon) #create mask
mask=np.logical_not(mask) #invert mask

####################### find the pattern center ##############
rc, cc = autocenter(im, mask=mask)
autocenter test.zip

ax1.scatter(cc, rc, color='r')

ax1.get_xaxis().set_visible(False)
ax1.get_yaxis().set_visible(False)
plt.tight_layout()
plt.show()

Mention baseline_dt() supports 2D data in the DOC

Version / Platform Info

• scikit-ued Version: 2.1.7
• Happens in latest version? (Yes/No): Yes
• Known Operating System(s) affected__:
  • Windows
  • Linux
  • Mac OS

Expected Behavior

In the DOC:

The baseline_dt() routine will usually be more accurate than its baseline_dwt() counterpart. However, baseline_dwt() can be applied to 1D and 2D data.

But, baseline_dt() also works for 2D data:

import numpy as np
from scipy import signal
from skued import baseline_dt

# create example data
t = np.linspace(-1, 1, 200, endpoint=False)
sig  = np.cos(2 * np.pi * 7 * t) + signal.gausspulse(t - 0.4, fc=2)
widths = np.arange(1, 31)
cwtmatr = signal.cwt(sig, signal.ricker, widths)

# get the baseline
baseline = baseline_dt(cwtmatr, wavelet = 'qshift3', level = 6, max_iter = 150)

# plot
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(6, 3))

ax1.imshow(cwtmatr)
ax2.imshow(baseline)
ax3.imshow(cwtmatr-baseline)

plt.tight_layout()

So, is it better to say both support 1D and 2D data?

silent relicensing to a viral license is not good

Since it will cause dependent packages to use the GPL v3-licensed package and to get infected.

IMHO: you should change package name, leaving scikit-ued MIT licensed, but unupdated, and use some other name for GPL-licensed successor, to prevent it from being automatically installed and used by non-GPL packages depending on scikit-ued.

Slow import

Importing scikit-ued is currently very slow. Timing the statement import skued currently gives me ~2.55s.

Since scikit-ued is a single-namespace package, all functions and classes are imported with the package. Therefore, we expect it to be somewhat slow. However, compared to NumPy (~160ms), which also has a large base namespace, we are still much slower.

powdersim reciprocal lattice vector normalization issue

Version / Platform Info

• scikit-ued Version: 2.1.13
• Happens in latest version? (Yes/No): Yes
• Known Operating System(s) affected:
  • Windows
  • Linux
  • Mac OS

Expected Behavior

The unit of the reciprocal lattice vector that is used in the function powdersim is expected to be 1/Å. (see the documentation).

Actual Behavior

The unit of $q$ in fact is $2\pi$/Å.

Minimal Example of Issue

I illustrated this issue in a jupyter notebook, where the result of scikit-ued is compared with the output of py4DSTEM.

where are the tutorials data sets?

beginner trying to run a tutorial at

https://scikit-ued.readthedocs.io/en/master/tutorials/baseline.html

a data set is loaded with the command
s, intensity = np.load("docs/tutorials/data/powder.npy")

I get an error
FileNotFoundError: [Errno 2] No such file or directory: 'docs/tutorials/data/powder.npy'

How do I download 'docs/tutorials/data/powder.npy'?
Or perhaps I already have but don't know the path.

I installed with anaconda3 on Windows 10 like this
(datashader) C:\Users\david>conda config --add channels conda-forge
(datashader) C:\Users\david>conda install scikit-ued

I copy and paste the example into a jupyter notebook, erasing the >>>'s

First stage dual tree filters

Version / Platform Info

• scikit-ued Version: 2.1.5
• Happens in latest version? (Yes/No): Yes
• Known Operating System(s) affected__:
  • Windows
  • Linux
  • Mac OS

Expected Behavior

Hello. My question is about the function dt_first_stage(wavelet) in scikit-ued/skued/baseline/dtcwt.py module. To my best knowledge, for the two sets of the first filter banks it is necessary to translate one set of filters by one sample with respect to the other set.

Actual Behavior

As one can see from the minimal example below, the first stage real and imaginary filters are actually same as the filters for the normal DWT. Furthermore, the first stage real and imaginary filters are the same. If this is really true, the DTCWT cannot be realized.

Minimal Example of Issue

The function dt_first_stage(wavelet) is actually copied from the source code.

from pywt import Wavelet

wavelet = Wavelet("db2")
real_first, imag_first = dt_first_stage("db2") 
real_first.filter_bank == wavelet.filter_bank  # return True
imag_first.filter_bank == wavelet.filter_bank  # return True
real_first.filter_bank == imag_first.filter_bank  # return True

Crystal orientation in kinematical single-crystal simulation

Hello,
I found scikit-ued very useful!

I would like to perform kinematical single-crystal simulation orienting the crystal to a different zone axis.
Could you improve the tutorial section with this case?
https://scikit-ued.readthedocs.io/en/master/tutorials/simulation.html

Best regards
Luca

baseline_dt doesn't work well for data with high spot and noisy background

Version / Platform Info

• scikit-ued Version: v2.1.7
• Happens in latest version? (Yes/No): Yes

Expected Behavior

Remove the noisy background and subtract it from the three high spots.

Something like this, but only three high spots:

import numpy as np
import matplotlib.pyplot as plt

data = np.load('test_baseline_dt.npy')
data_expected = np.where(data<1e-5, 0, data)
background_at_high_spot = 0.5e-5

fig, ax = plt.subplots(figsize=(12, 6))

m = ax.imshow(np.where(data_expected>0, data_expected-background_at_high_spot, data_expected))
plt.colorbar(m)

Actual Behavior

Only quite low values are detected as the baseline.

Minimal Example of Issue

import numpy as np
from skued import baseline_dt
import matplotlib.pyplot as plt

data = np.load('test_baseline_dt.npy')
# assign nan to quite large negative value
baseline = baseline_dt(np.nan_to_num(data, nan=-1e10), wavelet = 'qshift3', level=10, max_iter = 150)


# plot
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(12, 6))

m1 = ax1.imshow(data)
m2 = ax2.imshow(baseline, vmin=-5e-6)
m3 = ax3.imshow(data-baseline)

plt.colorbar(m1, ax=ax1, shrink=0.5)
plt.colorbar(m2, ax=ax2, shrink=0.5, extend='min')
plt.colorbar(m3, ax=ax3, shrink=0.5)

plt.tight_layout()

Data

test_baseline_dt.zip

The repo is too large

57MiB is too large for such a git repo. Usualy this happens if someone stores binaries in the repo. Consider moving the binaries out of the repo (for example wheels) and using git lfs for the ones which cannot be moved out. I insist on rewriting the history. https://github.com/bozaro/git-lfs-migrate and https://rtyley.github.io/bfg-repo-cleaner/ may be helpful. Removing binary dependencies in f7de365 won't help: the history still persists, the repo is still large.