Comments (7)
It most likely means that due to the sampling you have produced a few hmm's who live on different states (or at least not a subset) of the prior's states. I suggest you subselect the samples accordingly. It might even be enough to call bhmm_largest = bhmm.submodel_largest()
from deeptime.
It most likely means that due to the sampling you have produced a few hmm's who live on different states (or at least not a subset) of the prior's states. I suggest you subselect the samples accordingly. It might even be enough to call
bhmm_largest = bhmm.submodel_largest()
Thank you very much for your prompt reply!
When I finished running the command bhmm_largest = bhmm.submodel_largest(connectivity_threshold=0.2, dtrajs=dtrajs)
,it still gave me the same error, and I noticed that the notes on metastable_sets
in the manual read: "This is only recommended for visualization purposes. You cannot compute any actual quantity of the coarse-grained kinetics without employing the fuzzy memberships!"
`---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[301], line 8
6 for i in range(nstates):
7 for j in range(nstates):
----> 8 mfpt[i, j] = bhmm_largest.evaluate_samples('transition_model/mfpt', A=bhmm_largest.prior.metastable_sets[i], B=bhmm_largest.prior.metastable_sets[j]).mean
9 # mfpt[i, j] = bhmm_largest.gather_stats('transition_model/mfpt', A=bhmm_largest.prior.metastable_sets[i], B=bhmm_largest.prior.metastable_sets[j]).mean
11 inverse_mfpt = np.zeros_like(mfpt)
File ~/whshen/anaconda3/envs/workshop/lib/python3.9/site-packages/deeptime/base.py:238, in BayesianModel.evaluate_samples(self, quantity, delimiter, *args, **kwargs)
218 r""" Obtains a quantity (like an attribute or result of a method or a property) from each of the samples.
219 Returns as list.
220
(...)
235 A list of the quantity evaluated on each of the samples. If can be converted to float ndarray then ndarray.
236 """
237 from deeptime.util.stats import evaluate_samples as _eval
--> 238 return _eval(self.samples, quantity=quantity, delimiter=delimiter, *args, **kwargs)
File ~/whshen/anaconda3/envs/workshop/lib/python3.9/site-packages/deeptime/util/stats.py:189, in evaluate_samples(samples, quantity, delimiter, *args, **kwargs)
187 samples = [call_member(s, q) for s in samples]
188 if quantity is not None:
--> 189 samples = [call_member(s, quantity, *args, **kwargs) for s in samples]
190 try:
191 samples = np.asfarray(samples)
...
602 """
603 if np.max(A) > self.n_states:
--> 604 raise ValueError('Chosen set contains states that are not included in the active set.')
ValueError: Chosen set contains states that are not included in the active set.I think the problem may be here, I can't directly use
bhmm_largest.prior.metastable_sets[i]` to get MFPT.
To add, I use BayesianMSM to calculate MFPT on the same sample data, and I can get the result, but I don't know what is wrong with BayesianHMM.
from deeptime.
Yeah so I looked into and I should have seen it sooner 😆
the transition model is the -coarse grained- space, so when you are looking at "metastable_sets", this is in finegrained space, ie clustering space! now you are asking to compute mfpt over clustering-space states (which there are presumably many more than in coarse grained space), and it rightfully complains about it.
Try the following: If you are computing MFPT between coarse-grained set 0 and 1, you can call
bayesian_hmm.gather_stats('transition_model/mfpt', A=[0], B=[1]).mean
This already takes care of the fuzzy state assignment, as you are not operating in fine-grained space.
from deeptime.
By the way! We gave a workshop on deeptime/pyemma a while ago and also covered HMMs there. You can find the corresponding notebook here. Given a few free minutes I'll integrate it into the documentation here, I think. :)
https://github.com/markovmodel/pyemma-workshop/blob/master/notebooks/06-hmm.ipynb
from deeptime.
That's right, I understand now, but I have encountered another problem, please ask for your advice, it seems to be a problem with my data sample, the MFPT I calculated is too large. At the same time give me the following warning: LinAlgWarning: Ill-conditioned matrix (rcond=1.40001e-17): result may not be accurate.
But I can get reasonable-looking results with BayesianMSM, which makes me very puzzled.
from deeptime.
It could be that the transitions are not sampled well enough, but I am guessing here. Have you tried other combinations of metastable states? It could also be a problem with your clustering, projection method, featurization.... MSM and HMM estimation can be tricky. I suggest you methodically check everything and also score it with eg VAMP-2 score. And yeah, that rcond number does not inspire confidence. You could also look at your transition matrix (of the CG matrix) to see how it looks like. And visualizing the population of data frames onto a 2d projection of your data (don't forget to check all relevant projections that fall out of the projection method of your choice). Relevance is correlated to the singular (or eigen) value of the projection component.
from deeptime.
And if you get reasonable results with bayesian msm you may have to tweak the estimation parameters a little when it comes to the bhmm. The prior is estimated with lower precision than the 'normal' bhmm. Also the mfpt of the prior itself would be interesting. is it also ill conditioned?
from deeptime.
Related Issues (20)
- Cannot install deeptime. I have Python 3.10 (Windows 10) and trying to install using Visual Studio 2022 HOT 12
- Error when installing from local repo via pip HOT 4
- TICA fit-once transform-many HOT 6
- Automate generating bdist wheels HOT 1
- Allowing weights for VAMP dimensionality reduction HOT 6
- Error importing deeptime with scikit-learn 1.2.0 HOT 4
- About applying Deeptime to molecular dynamics simulation trajectories on protien HOT 3
- Handling parallel simulations HOT 1
- Implied timescales from deeptime.decomposition.TICA HOT 2
- [Bug] At least one of the given states is not contained in this model HOT 2
- VAMPnet gives highly fluctuating score results HOT 1
- Bayesian Markov models give different MFPT
- Installation without conda and pip HOT 3
- Which version of VAMPNets is there? HOT 1
- deeptime cannot be installed on Mac computers with Apple Silicon and Python 3.12.0. HOT 4
- Does one need identical features to compare between related systems? HOT 4
- Example/tutorial for dealing with MD trajectories HOT 1
- Applying TICA to ragged feature vectors HOT 2
- Cross Validation score for MaximumLikelihoodMSMs and BayesianMSMs HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deeptime.