laurentrdc/scikit-ued

Collection of algorithms and routines for (ultrafast) electron diffraction and scattering

scikit-ued's Issues

baseline_dt doesn't work well for data with high spot and noisy background

Version / Platform Info

• scikit-ued Version: v2.1.7
• Happens in latest version? (Yes/No): Yes

Expected Behavior

Remove the noisy background and subtract it from the three high spots.

Something like this, but only three high spots:

import numpy as np
import matplotlib.pyplot as plt

data = np.load('test_baseline_dt.npy')
data_expected = np.where(data<1e-5, 0, data)
background_at_high_spot = 0.5e-5

fig, ax = plt.subplots(figsize=(12, 6))

m = ax.imshow(np.where(data_expected>0, data_expected-background_at_high_spot, data_expected))
plt.colorbar(m)

Actual Behavior

Only quite low values are detected as the baseline.

Minimal Example of Issue

import numpy as np
from skued import baseline_dt
import matplotlib.pyplot as plt

data = np.load('test_baseline_dt.npy')
# assign nan to quite large negative value
baseline = baseline_dt(np.nan_to_num(data, nan=-1e10), wavelet = 'qshift3', level=10, max_iter = 150)


# plot
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(12, 6))

m1 = ax1.imshow(data)
m2 = ax2.imshow(baseline, vmin=-5e-6)
m3 = ax3.imshow(data-baseline)

plt.colorbar(m1, ax=ax1, shrink=0.5)
plt.colorbar(m2, ax=ax2, shrink=0.5, extend='min')
plt.colorbar(m3, ax=ax3, shrink=0.5)

plt.tight_layout()

Data

test_baseline_dt.zip

Slow import

Importing scikit-ued is currently very slow. Timing the statement import skued currently gives me ~2.55s.

Since scikit-ued is a single-namespace package, all functions and classes are imported with the package. Therefore, we expect it to be somewhat slow. However, compared to NumPy (~160ms), which also has a large base namespace, we are still much slower.

where are the tutorials data sets?

beginner trying to run a tutorial at

https://scikit-ued.readthedocs.io/en/master/tutorials/baseline.html

a data set is loaded with the command
s, intensity = np.load("docs/tutorials/data/powder.npy")

I get an error
FileNotFoundError: [Errno 2] No such file or directory: 'docs/tutorials/data/powder.npy'

How do I download 'docs/tutorials/data/powder.npy'?
Or perhaps I already have but don't know the path.

I installed with anaconda3 on Windows 10 like this
(datashader) C:\Users\david>conda config --add channels conda-forge
(datashader) C:\Users\david>conda install scikit-ued

I copy and paste the example into a jupyter notebook, erasing the >>>'s

powdersim reciprocal lattice vector normalization issue

Version / Platform Info

• scikit-ued Version: 2.1.13
• Happens in latest version? (Yes/No): Yes
• Known Operating System(s) affected:
  • Windows
  • Linux
  • Mac OS

Expected Behavior

The unit of the reciprocal lattice vector that is used in the function powdersim is expected to be 1/Å. (see the documentation).

Actual Behavior

The unit of $q$ in fact is $2\pi$/Å.

Minimal Example of Issue

I illustrated this issue in a jupyter notebook, where the result of scikit-ued is compared with the output of py4DSTEM.

silent relicensing to a viral license is not good

Since it will cause dependent packages to use the GPL v3-licensed package and to get infected.

IMHO: you should change package name, leaving scikit-ued MIT licensed, but unupdated, and use some other name for GPL-licensed successor, to prevent it from being automatically installed and used by non-GPL packages depending on scikit-ued.

Mention baseline_dt() supports 2D data in the DOC

Version / Platform Info

• scikit-ued Version: 2.1.7
• Happens in latest version? (Yes/No): Yes
• Known Operating System(s) affected__:
  • Windows
  • Linux
  • Mac OS

Expected Behavior

In the DOC:

The baseline_dt() routine will usually be more accurate than its baseline_dwt() counterpart. However, baseline_dwt() can be applied to 1D and 2D data.

But, baseline_dt() also works for 2D data:

import numpy as np
from scipy import signal
from skued import baseline_dt

# create example data
t = np.linspace(-1, 1, 200, endpoint=False)
sig  = np.cos(2 * np.pi * 7 * t) + signal.gausspulse(t - 0.4, fc=2)
widths = np.arange(1, 31)
cwtmatr = signal.cwt(sig, signal.ricker, widths)

# get the baseline
baseline = baseline_dt(cwtmatr, wavelet = 'qshift3', level = 6, max_iter = 150)

# plot
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(6, 3))

ax1.imshow(cwtmatr)
ax2.imshow(baseline)
ax3.imshow(cwtmatr-baseline)

plt.tight_layout()

So, is it better to say both support 1D and 2D data?

Crystal orientation in kinematical single-crystal simulation

Hello,
I found scikit-ued very useful!

I would like to perform kinematical single-crystal simulation orienting the crystal to a different zone axis.
Could you improve the tutorial section with this case?
https://scikit-ued.readthedocs.io/en/master/tutorials/simulation.html

Best regards
Luca

autocenter() fails with some SAED pattern

Version / Platform Info

• scikit-ued Version: 2.1.7
• Happens in latest version? (Yes/No): yes
• Known Operating System(s) affected__:
  • [ x] Windows
  • Linux
  • Mac OS

Expected Behavior

autocenter() should find the pattern center

Actual Behavior

It results a wrong pattern center

Minimal Example of Issue

from skued import diffread, autocenter
import numpy as np
import matplotlib.pyplot as plt
from skimage.draw import polygon2mask

im = diffread('SAED_pattern.dm3')

fig, ax1 = plt.subplots(figsize = (4.5, 4.5))
ax1.imshow(im, cmap='gray', vmin=0, vmax=1000)

########## mask the beam stopper ########
polygon=np.array([[ 821.06421356, 1232.58513709],
[1128.41197691, 1368.52741703],
[1737.1969697 , 9.1046176 ],
[1347.1017316 , 9.1046176 ]])
mask = polygon2mask(im.shape, polygon) #create mask
mask=np.logical_not(mask) #invert mask

####################### find the pattern center ##############
rc, cc = autocenter(im, mask=mask)
autocenter test.zip

ax1.scatter(cc, rc, color='r')

ax1.get_xaxis().set_visible(False)
ax1.get_yaxis().set_visible(False)
plt.tight_layout()
plt.show()

First stage dual tree filters

Version / Platform Info

• scikit-ued Version: 2.1.5
• Happens in latest version? (Yes/No): Yes
• Known Operating System(s) affected__:
  • Windows
  • Linux
  • Mac OS

Expected Behavior

Hello. My question is about the function dt_first_stage(wavelet) in scikit-ued/skued/baseline/dtcwt.py module. To my best knowledge, for the two sets of the first filter banks it is necessary to translate one set of filters by one sample with respect to the other set.

Actual Behavior

As one can see from the minimal example below, the first stage real and imaginary filters are actually same as the filters for the normal DWT. Furthermore, the first stage real and imaginary filters are the same. If this is really true, the DTCWT cannot be realized.

Minimal Example of Issue

The function dt_first_stage(wavelet) is actually copied from the source code.

from pywt import Wavelet

wavelet = Wavelet("db2")
real_first, imag_first = dt_first_stage("db2") 
real_first.filter_bank == wavelet.filter_bank  # return True
imag_first.filter_bank == wavelet.filter_bank  # return True
real_first.filter_bank == imag_first.filter_bank  # return True

The repo is too large

57MiB is too large for such a git repo. Usualy this happens if someone stores binaries in the repo. Consider moving the binaries out of the repo (for example wheels) and using git lfs for the ones which cannot be moved out. I insist on rewriting the history. https://github.com/bozaro/git-lfs-migrate and https://rtyley.github.io/bfg-repo-cleaner/ may be helpful. Removing binary dependencies in f7de365 won't help: the history still persists, the repo is still large.