chuckie82/skopi

skopi's Issues

skopi.png

Issues found during the execution of some example files

Issues are found during the execution of some example files under the examples/scripts directory.

Major issues

cspi_generate_synthetic_dataset.py

Run:

python -i cspi_generate_synthetic_dataset.py --config cspi_generate_synthetic_dataset_config.json --dataset 3iyf

Error:

ad PDB: /reg/neh/home5/deebanr/rdeeban-pysingfel/pysingfel/examples/input/pdb/3iyf.pdb
Load beam parameters: /reg/neh/home5/deebanr/rdeeban-pysingfel/pysingfel/examples/input/pdb/3iyf.pdb
Load detector geometry: /reg/neh/home5/deebanr/rdeeban-pysingfel/pysingfel/examples/input/lcls/amo86615/PNCCD::CalibV1/Camp.0:pnCCD.1/geometry/0-end.data
Traceback (most recent call last):
  File "cspi_generate_synthetic_dataset.py", line 217, in <module>
    main()
  File "cspi_generate_synthetic_dataset.py", line 104, in main
    det = PnccdDetector(geom=geom_file, beam=beam)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/detector/pnccd.py", line 9, in __init__
    super(PnccdDetector, self).__init__(*args, **kwargs)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/detector/lcls.py", line 67, in __init__
    self.initialize_pixels_with_beam(beam=beam)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/detector/base.py", line 139, in initialize_pixels_with_beam
    intensity = beam.get_photons_per_pulse() / beam.get_focus_area()
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/beam/base.py", line 203, in get_photons_per_pulse
    return self._n_phot
AttributeError: 'Beam' object has no attribute '_n_phot'

Possible cause:

This line doesn't get a chance to run.

Example3D_MPI_HDF5.py

Run:

mpirun -n 2 python Example3D_MPI_HDF5.py --pdb=../input/3iyf.pdb --geom=../lcls/amo86615/PNCCD::CalibV1/Camp.0:pnCCD.1/geometry/0-end.data --beam=../input/exp_chuck.beam --numSlices=1 --UniformOrientation=1 --outDir=/path/to/outdir --savePhotons=0

Error/Dependency issue:

Traceback (most recent call last):
  File "Example3D_MPI_HDF5.py", line 10, in <module>
    import Bio
ModuleNotFoundError: No module named 'Bio'
Traceback (most recent call last):
  File "Example3D_MPI_HDF5.py", line 10, in <module>
    import Bio
ModuleNotFoundError: No module named 'Bio'

ExampleAggregate.py

Run:

python ExampleAggregate.py

Error:

Traceback (most recent call last):
  File "ExampleAggregate.py", line 50, in <module>
    det = PnccdDetector(geom=geom, beam=beam)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/detector/pnccd.py", line 9, in __init__
    super(PnccdDetector, self).__init__(*args, **kwargs)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/detector/lcls.py", line 63, in __init__
    self.initialize(geom=geom, run_num=run_num, cframe=cframe)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/detector/lcls.py", line 91, in initialize
    temp = [xp.asarray(t) * 1e-6 for t in self.geometry.get_pixel_coords(cframe=cframe)]
TypeError: 'NoneType' object is not iterable

ExampleHydrocarbons.py

Missing file:

Traceback (most recent call last):
  File "ExampleHydrocarbons.py", line 17, in <module>
    particleC.read_pdb(os.path.join(pwd,'../input/pdb/cyclohexane.pdb'), ff='WK')
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/particle.py", line 233, in read_pdb
    atoms,atomslist = symmpdb(fname, ff)
  File "/cds/home/c/cwang31/.local/lib/python3.7/site-packages/skopi/util.py", line 154, in symmpdb
    fin = open(fname, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '../input/pdb/cyclohexane.pdb'

plot_positions.py

Run:

python plot_positions.py

Error

Traceback (most recent call last):
  File "plot_positions.py", line 29, in <module>
    states, coords = distribute_particles(particles={particleOp:numOpen,particleCl:numClosed}, beam_focus_radius=beam._focus_xFWHM/2, jet_radius=1e-4)
TypeError: distribute_particles() got an unexpected keyword argument 'beam_focus_radius'

Minor issues

gui.py

• Here is an okay-but-not-so-conventionally-used string concatenation. It's probably better to use a single line of code since it's just a path.

ExampleMPI.py

No example found in the code that runs it.

Feature request: calculate theoretical number of snapshots required

import numpy as np

def logFactorial(n):
    if n < 20:
        value = np.log(np.math.factorial(n))
    else:
        value = 0.5*np.log(2*np.pi*n) + n*np.log(n/np.e)
    return value

def pnm(p,N,M):
#this function gives the probability to observe a voxel at least M times
#from an ensemble of N snapshots
# p = probability to hit a voxel for a single snapshot
# N = Number of Snapshots
# M = Redundancy
    if N < M:
        s = 1
    else:
        s = 0
        lp = np.log(p)
        lmp = np.log(1-p)
        for k in np.arange(M):
            s = s + np.exp(logFactorial(N) - logFactorial(N-k) - logFactorial(k) + k*lp + (N-k)*lmp)
    return np.maximum(1-s,0)

def numberOfSnapShots(d,D,nPhotons,SNR,P_tilde):
    #nS number of Snapshots
    #d is the Resolution
    # D is the Diameter of a Molecule
    #nPhotons is the Number of Photons
    #P_tilde is the combined probability
    #Number of Resolution elements
    R = D/d
    #number of voxels at Resolution Shell
    nV_Shell = 16*np.pi*R**2
    #probability per Shannon voxel
    p = 1./(4*R)
    M = np.ceil(SNR**2/nPhotons)
    # P -> Probability to observe a voxel at least M times from an ensemble of nS snapshot
    # obtained from given P_tilde
    P = np.exp(2*np.log(P_tilde)/nV_Shell)
    nSmax = 1e6#
    step = 2**10#
    nS0 = M
    while step > 1:
        for nS in np.arange(nS0,nSmax,step):
            prob = pnm(p,nS,M)
            if prob > P:
                break
        nS0 = nS - step
        step = step /2
    return nS

d = 14 # resolution (Angstrom)
D = 70 # Angstrom diameter (Angstrom)
nPhotons = 0.0076 # no. of photons per Shannon pixel at desired res d
P_tilde = 0.5
SNR=1
nS = numberOfSnapShots(d,D,nPhotons,SNR,P_tilde)
print("number of snapshots required: ", nS)

uniform quaternion generation computationally uneven across different numbers of orientations

We noticed a computational anomaly in skopi.get_uniform_quat(N_orientations, True). It spends far less time to do this for 64k or even 256k orientations compared to 8k, 16k, 32k and 128k. Any idea why? Thanks.

def doquat(num):
... start = time.time()
... skp.get_uniform_quat(num,True)
... end = time.time()
... elapsed = end - start
... return elapsed
...
doquat(1024)
0.013123035430908203
doquat(2048)
0.030680179595947266
doquat(10248)
22.523311614990234
doquat(102416)
44.15811800956726
doquat(102432)
86.49000382423401
doquat(102464)
0.8508529663085938
doquat(1024128)
337.60426592826843
doquat(1024256)
1.994347095489502

P.S. I also submitted this to the lcls-psana/skopi repo since wasn't sure if all the authors were watching this repo or that one.

Support the AGIPD detector

Dear all,

I would like to contribute to add support to the AGIPD detector, is there already a plan for it? If not, could you please let me know where to start? Would it be possible to set a detector geometry in SKOPI from a CrrystFEL geometry file directly?

Many thanks,
Jun

numba

Tried to run the example script but got TypeError.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-aa8f1eac6d96> in <module>
     38 tic = time.time()
     39 experiment = sk.SPIExperiment(det, beam, particle)
---> 40 img = experiment.generate_image()
     41 toc = time.time()
     42 print(">>> It took {:.2f} seconds to finish SPI calculation.".format(toc-tic))

~/skopi/skopi/experiment/base.py in generate_image(self, return_orientation)
     36             return self.det.assemble_image_stack(img_stack), orientation
     37         else:
---> 38             img_stack = self.generate_image_stack()
     39             return self.det.assemble_image_stack(img_stack)
     40 

~/skopi/skopi/experiment/base.py in generate_image_stack(self, return_photons, return_intensities, return_orientation, always_tuple)
     74             for i, particle_group in enumerate(sample_state):
     75                 group_complex_pattern += self._generate_group_complex_pattern(
---> 76                     recidet, i, particle_group)
     77 
     78             group_pattern = np.abs(group_complex_pattern)**2

~/skopi/skopi/experiment/base.py in _generate_group_complex_pattern(self, recidet, i, particle_group)
    108             pixel_momentum=recidet.pixel_position_reciprocal,
    109             orientations=orientations,
--> 110             inverse=True)
    111 
    112         for j, position in enumerate(positions):

~/skopi/skopi/geometry/slice_.py in take_n_slices(volume, voxel_length, pixel_momentum, orientations, inverse)
    124     for l in range(slice_num):
    125         slices_holder[l] = take_slice(volume, voxel_length, pixel_momentum,
--> 126                                       orientations[l], inverse)
    127 
    128     return slices_holder

~/skopi/skopi/geometry/slice_.py in take_slice(volume, voxel_length, pixel_momentum, orientation, inverse)
     75     """
     76     # construct the rotation matrix
---> 77     rot_mat = convert.quaternion2rot3d(orientation)
     78     if inverse:
     79         rot_mat = np.linalg.inv(rot_mat)

TypeError: expected dtype object, got 'numpy.dtype[float64]'

There seems to be issues with numba.
numba/numba#6041

ExampleSPI.py

load_data.png

numberOfSnapshots.png