Comments (5)
Here is the code with a sample file that I used:
github_issue_file.txt
sample_df.csv
from msm.
Looking at that qmatrix
and ematrix
, that model will not be identifiable from the data. You are allowing misclassification between all states, and instantaneous transitions between all states. There will be no way to tell from the data which combinations of parameter values are more plausible. The likelihood will be a flat function of the parameters, so that the optimum doesn't exist.
I'd advise to consider the structure of the process that you are modelling, and choose the allowed transitions and misclassifications more carefully. These are continuous-time Markov models - so even a transition between a pair of states can happen over an interval, it may not be possible in continuous time - see the course notes. Also make sure the observation scheme is set appropriately (through the obstype
or exacttimes
arguments) - do you know when the transitions happen in continuous time?
Don't give up with using fixed parameter values - e.g. if you must model misclassification, then this may only be possible with fixed misclassification probabilities.
from msm.
Dear Prof. Jackson,
Thanks so much for your suggestions, it really seems like I might have allowed too many possibilities for the model.
Could you explain and give me an example of how to specify fixedpars
for the misclassification probabilities? In the documentation it says:
" These are given in the order: transition intensities
(reading across rows of the transition matrix), covariates on intensities (ordered
by intensities within covariates), hidden Markov model parameters, including
misclassification probabilities or parameters of HMM outcome distributions (ordered by parameters within states), hidden Markov model covariate parameters
(ordered by covariates within parameters within states), initial state occupancy
probabilities (excluding the first probability, which is fixed at one minus the sum
of the others)."
If I have 4 possible states, how should the misclassification probabilities be specified such that not the transition intensities are fixed instead? And do they change in case I use covariates?
Thanks a lot in advance!
from msm.
Can I explain in the simplest case with a 2-state model? If you have, say qmatrix = rbind(c(0, 0.01), c(1, 0.02))
, and ematrix = rbind(c(0, 0.2), c(0.1, 0))
, there are 4 parameters in the model. These parameters are given labels of 1, 2, 3, 4 to refer to them in fixedpars
.
So to fix both misclassification probabilities, but not the intensities, specify fixedpars = c(3, 4)
. Or to fix the first misclassification probability at 0.2, but estimate the second one (starting from an initial value of 0.1), it will be just fixedpars=3
.
Does the general idea behind the text in the help file make more sense now? This text explains the order that the parameters are in when referring to them as 1,2,3,4,...
from msm.
Thank you for your explanation, that helped a lot. I was able - after a bit of trial and error - to set the parameters accordingly (fixing two of the misclassification parameters) such that the models converged. I also restricted my initial qmatrix with the allowed transitions a bit more, as well as the misclassification matrix. All together helped, so thanks a lot!
For anyone else reading this post: I want to give you an example for the fixedpars
arguments to showcase how it would work with covariates - as far as I understood it. Please correct me if I am wrong!
qmatrix <- rbind(
c(0, 0.1, 0, 0),
c(0.1, 0, 0.1, 0),
c(0.1, 0.1, 0, 0.1),
c(0.1, 0.1, 0, 0)
)
- qmatrix (4x4) with 8 parameters --> indices 1:8 for
fixedpars
would fix these qmatrix entries
covariates <- "~age + sex + bmi"
- 3 covariates, on 8 parameters from the qmatrix (3*8)--> indices 9:32 for
fixedpars
would fix the covariates on the intensities
ematrix <- rbind(
c(0, 0.1, 0, 0),
c(0.1, 0, 0.1, 0.1),
c(0, 0.1, 0, 0.1),
c(0, 0, 0, 0)
)
- misclassification matrix (4x4) with 6 parameters --> indices 33:38 would fix the misclassification rates
Cheers!
from msm.
Related Issues (20)
- Guidance Request on Addressing Transition Probability Estimation Issue in msm Package HOT 4
- multiple covariates HOT 7
- The question of msm HOT 5
- Implementing a semi-Markov model using 'msm' package HOT 5
- A question about very high total times using msm.totlos HOT 14
- Error in estimating transition intensities HOT 2
- Censored states in simfitted.msm HOT 1
- People with one observation shouldn't be dropped in hidden Markov models HOT 1
- Fitting simple healthy-ill-death model without recovery HOT 12
- prevalence.msm censtime question HOT 4
- Pearson goodness of fit tests fail for models with covariate interactions HOT 1
- Implementation of Markov polymorphism model with 5 states and 8 paths HOT 9
- How does MSM handle missing covariate data? HOT 1
- initial value in 'vmmin' is not finite HOT 2
- Observations within subjects ... are not ordered by time HOT 2
- Too many paths causing errors? HOT 2
- ns() not handled
- Question: CI for transition intensities ratios HOT 2
- pmatrix.msm output same matrix for time-inhomogeneous MV HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from msm.