mczwier/westpa_py3

WESTPA: The Weighted Ensemble Simulation Toolkit with Parallelization and Analysis

westpa_py3's Issues

Analysis Tools Issues

So I've gone ahead and pulled in all these changes into a branch of my own fork so that I can test and make adjustments to analysis tools as necessary.

I've already gotten through a quick few bugs with some of the stuff I wrote (w_ipa, for instance; python changes how it does the string encodings).

I just figured this might be a good place to document some of the issues I'm running into. I'll make a pull request once I've fixed any that pop up, but.

  File "/westpa/lib/west_tools/w_assign.py", line 468, in go
    self.output_file.create_dataset('bin_labels', data=labels, compression=9)
  File "/home/astatide/.local/lib/python3.7/site-packages/h5py/_hl/group.py", line 136, in create_dataset
    dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
  File "/home/astatide/.local/lib/python3.7/site-packages/h5py/_hl/dataset.py", line 118, in make_new_dset
    tid = h5t.py_create(dtype, logical=1)
  File "h5py/h5t.pyx", line 1630, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1652, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1713, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U17')

Looks like there are some underlying changes in some of the libraries regarding how strings are encoded and handled; I'll try changing how the labels dataset is created and going from there.

Keep in mind I am running into the problem coming from within w_ipa; this shouldn't be a problem, but if anyone has successfully run w_assign that would be good info!

ImportError: dynamic module does not define module export function (PyInit__assign)

Hello,

I'm testing this out on one of my systems and cannot seem to get passed the initialization step. I get the following error:

Warning: When compiling code please add the following flags to nvcc:
-gencode arch=compute_35,code=[compute_35,sm_35]
-gencode arch=compute_61,code=[compute_61,sm_61]
-gencode arch=compute_70,code=[compute_70,sm_70]
simulation py3_test root is /bgfs/ltc1/home/atb43/qmmm/september/6_WESTPA/py3_test
Traceback (most recent call last):
File "/bgfs/ltc1/home/atb43/qmmm/py3_westpa/lib/cmds/w_init.py", line 28, in
import westpa
File "/bgfs/ltc1/home/atb43/qmmm/py3_westpa/lib/west_tools/westpa/init.py", line 1, in
from . import _rc
File "/bgfs/ltc1/home/atb43/qmmm/py3_westpa/lib/west_tools/westpa/_rc.py", line 27, in
from .yamlcfg import YAMLConfig
File "/bgfs/ltc1/home/atb43/qmmm/py3_westpa/lib/west_tools/westpa/yamlcfg.py", line 37, in
from westpa.binning import NopMapper
File "/bgfs/ltc1/home/atb43/qmmm/py3_westpa/lib/west_tools/westpa/binning/init.py", line 1, in
from . import _assign
ImportError: dynamic module does not define module export function (PyInit__assign)

This didn't happen with the python2 WESTPA. I thought this may be due to me defining my bins outside of the west.cfg (in the system.py) and so I tried doing that, but get the same error.

Here is my west.cfg file:

The master WEST configuration file for a simulation.

vi: set filetype=yaml :

---

west:
system:
driver: system.SDASystem
module_path: $WEST_SIM_ROOT
propagation:
max_total_iterations: 1000
max_run_wallclock: 72:00:00
propagator: executable
gen_istates: true
block_size: 1
data:
west_data_file: west.h5
datasets:
- name: pcoord
scaleoffset: 4
- name: solute_coords
scaleoffset: 4
- name: dihedral
scaleoffset: 4
- name: c5n34
scaleoffset: 4
- name: c7n34
scaleoffset: 4
- name: c9n34
scaleoffset: 4
- name: c12n34
scaleoffset: 4
- name: c14n34
scaleoffset: 4
- name: c16n34
scaleoffset: 4
- name: c21n34
scaleoffset: 4
- name: c25n34
scaleoffset: 4
- name: c23n34
scaleoffset: 4
data_refs:
segment: $WEST_SIM_ROOT/traj_segs/{segment.n_iter:06d}/{segment.seg_id:06d}
basis_state: $WEST_SIM_ROOT/bstates/{basis_state.auxref}
initial_state: $WEST_SIM_ROOT/istates/{initial_state.iter_created}/{initial_state.state_id}.rst
plugins:
executable:
environ:
PROPAGATION_DEBUG: 1
datasets:
- name: solute_coords
enabled: true
- name: dihedral
enabled: true
- name: c5n34
enabled: true
- name: c7n34
enabled: true
- name: c9n34
enabled: true
- name: c12n34
enabled: true
- name: c14n34
enabled: true
- name: c16n34
enabled: true
- name: c21n34
enabled: true
- name: c25n34
enabled: true
- name: c23n34
enabled: true
propagator:
executable: $WEST_SIM_ROOT/westpa_scripts/runseg.sh
stdout: $WEST_SIM_ROOT/seg_logs/{segment.n_iter:06d}-{segment.seg_id:06d}.log
stderr: $WEST_SIM_ROOT/slurm.err
stdin: null
cwd: null
environ:
SEG_DEBUG: 1
get_pcoord:
executable: $WEST_SIM_ROOT/westpa_scripts/get_pcoord.sh
stdout: $WEST_SIM_ROOT/job_logs/pcoord.log
stderr: $WEST_SIM_ROOT/job_logs/pcoord.err
gen_istate:
executable: $WEST_SIM_ROOT/westpa_scripts/gen_istate.sh
stdout: $WEST_SIM_ROOT/job_logs/istate.log
stderr: $WEST_SIM_ROOT/job_logs/istate.err
post_iteration:
enabled: true
executable: $WEST_SIM_ROOT/westpa_scripts/post_iter.sh
stdout: $WEST_SIM_ROOT/job_logs/post_iter.log
stderr: $WEST_SIM_ROOT/job_logs/post_iter.err
pre_iteration:
enabled: false
executable: $WEST_SIM_ROOT/westpa_scripts/pre_iter.sh
stderr: stdout

Settings for w_ipa, an interactive analysis program that can also automate analysis.

analysis:
directory: ANALYSIS # specify the directory all analysis files should exist in.
kinetics: # general options for both kinetics routines.
step_iter: 1
evolution: cumulative
extra: [ 'disable-correl' ]
analysis_schemes: # Analysis schemes. Required: name (TEST), states, and bins
DEFAULT:
enabled: True
bins:
- type: RectilinearBinMapper
boundaries:
- [0.0, 1.6, 'inf']
states:
- label: unbound
coords:
- [20]
- label: bound
coords:
- [1.56]

And here is my system.py file:

#!/usr/bin/env python

import numpy
import west
import os
from west import WESTSystem
from westpa.binning import RectilinearBinMapper
from westpa.binning import FuncBinMapper
from westpa.binning import RecursiveBinMapper
import logging

log = logging.getLogger(name)
log.debug('loading module %r' % name)

def radial_map(coords, mask, output):
center = numpy.array((1.6, 1.6))
coords -= center
quotients = numpy.divide(coords[:,1],coords[:,0])
angles = numpy.arctan(quotients)/(2*numpy.pi)*360
bins = numpy.arange(0,91,1)
output[mask] = numpy.digitize(angles, bins)[mask]
return output
class SDASystem(WESTSystem):
'''
Class specify binning schemes, walker counts, and other core weighted
ensemble parameters.
'''
def initialize(self):
self.pcoord_ndim = 2
self.pcoord_len = 2
self.pcoord_dtype = numpy.float32

    outer_mapper = RectilinearBinMapper(
            [[0, 1.6, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 'inf'],
             [0, 1.6, 4, 5, 'inf']]
                                        )

inner_mapper = RectilinearBinMapper(

[]

)

    radial_mapper = FuncBinMapper(radial_map, nbins=91)


    self.bin_mapper = RecursiveBinMapper(outer_mapper)
    self.bin_mapper.add_mapper(
            radial_mapper, 
            [1.6,1.6]
                               )

    self.bin_target_counts = numpy.empty((self.bin_mapper.nbins,), 
                                         dtype=numpy.int)
    self.bin_target_counts[...] = 5

Thanks,
Anthony

confirmed working: intermediate_p53 tutorial

I ran through the p53 example following the instructions on the LiveCoMS manuscript. The westpa dynamics part works and so did the analysis part.

Analysis tools broken for advanced_chignolin example

The westpa dynamics part for both WE_folding and WE_unfolding completed. However, running w_flxanl gave these errors:

[[email protected] WE_folding]$w_fluxanl
Calculating mean flux and confidence intervals for iterations [1,1001)
Traceback (most recent call last):
  File "/ihome/sam/kimwong/CRC/Lillian_Chong/2019_WESTPA-REVO/westpa_py3/lib/west_tools/w_fluxanl.py", line 335, in <module>
    WFluxanlTool().main()
  File "/ihome/sam/kimwong/CRC/Lillian_Chong/2019_WESTPA-REVO/westpa_py3/lib/west_tools/westtools/core.py", line 128, in main
    self.go()
  File "/ihome/sam/kimwong/CRC/Lillian_Chong/2019_WESTPA-REVO/westpa_py3/lib/west_tools/w_fluxanl.py", line 329, in go
    self.calc_store_flux_data()
  File "/ihome/sam/kimwong/CRC/Lillian_Chong/2019_WESTPA-REVO/westpa_py3/lib/west_tools/w_fluxanl.py", line 249, in calc_store_flux_data
    h5io.label_axes(target_group['flux'], ['n_iter'], units=['tau^-1'])
  File "/ihome/sam/kimwong/CRC/Lillian_Chong/2019_WESTPA-REVO/westpa_py3/lib/west_tools/westpa/h5io.py", line 230, in label_axes
    h5object.attrs['axis_labels'] = numpy.array(list(map(str,labels)))
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/ihome/sam/kimwong/install/python/anaconda3.7-2019.07/lib/python3.7/site-packages/h5py/_hl/attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "/ihome/sam/kimwong/install/python/anaconda3.7-2019.07/lib/python3.7/site-packages/h5py/_hl/attrs.py", line 177, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1630, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1652, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1713, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U6')
[[email protected] WE_folding]$

Analysis tools errors

Hello,

I've tried out some of the analysis tools as well with my system's h5 file. w_ipa does not work, I get the following error:

Welcome to w_ipa (WESTPA Interactive Python Analysis) v. 1.0B!
Run w.introduction for a more thorough introduction, or w.help to see a list of options.
Running analysis & loading files.
Traceback (most recent call last):
File "/home/atb43/Documents/py3_westpa/lib/west_tools/w_ipa.py", line 777, in
w.main()
File "/home/atb43/Documents/py3_westpa/lib/west_tools/westtools/core.py", line 176, in main
self.go()
File "/home/atb43/Documents/py3_westpa/lib/west_tools/w_ipa.py", line 697, in go
self.analysis_structure()
File "/home/atb43/Documents/py3_westpa/lib/west_tools/w_ipa.py", line 300, in analysis_structure
new_hash = self.hash_args(args=args, path=path, extra=[int(self.niters), pickle.dumps(assign.binning.mapper).encode('base64'), str(assign.states).encode('base64')])
AttributeError: 'bytes' object has no attribute 'encode'

Similarly, w_assign also does not work. Here is what I get:

Traceback (most recent call last):
File "/home/atb43/Documents/py3_westpa/lib/west_tools/w_assign.py", line 560, in
WAssign().main()
File "/home/atb43/Documents/py3_westpa/lib/west_tools/westtools/core.py", line 173, in main
self.make_parser_and_process()
File "/home/atb43/Documents/py3_westpa/lib/west_tools/westtools/core.py", line 154, in make_parser_and_process
self.process_all_args(args)
File "/home/atb43/Documents/py3_westpa/lib/west_tools/westtools/core.py", line 80, in process_all_args
fn(self,args)
File "/home/atb43/Documents/py3_westpa/lib/west_tools/w_assign.py", line 289, in process_args
self.binning.process_args(args)
File "/home/atb43/Documents/py3_westpa/lib/west_tools/westtools/binning.py", line 355, in process_args
self.mapper, self.mapper_pickle, self.mapper_hash = mapper_from_hdf5(self.mapper_source_group, self.mapper_source_hash)
File "/home/atb43/Documents/py3_westpa/lib/west_tools/westtools/binning.py", line 90, in mapper_from_hdf5
mapper = pickle.loads(pkldat)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xae in position 0: ordinal not in range(128)

Here is the analysis portion of my west.cfg:

analysis:
directory: ANALYSIS # specify the directory all analysis files should exist in.
kinetics: # general options for both kinetics routines.
step_iter: 1
evolution: cumulative
extra: [ 'disable-correl' ]
analysis_schemes: # Analysis schemes. Required: name (TEST), states, and bins
DEFAULT:
enabled: True
bins:
- type: RectilinearBinMapper
boundaries:
- [0.0, 1.6, 'inf']
states:
- label: unbound
coords:
- [20]
- label: bound
coords:
- [1.56]

I didn't test w_direct, since w_assign did not work.
The following tools work just fine:

• w_succ
• w_trace
• w_pdist
• plothist

Thanks,
Anthony

Consolidate paths

The current file layout for WESTPA is a total mess as a result of originally having been three separate repositories. This can be greatly simplified and improved now that everything lives in the same repo.

Replace imp module with importlib

The imp module is deprecated in Python 3.7. lib/west_tools/westpa/ext_loader.py should be updated.