kseverso/discovery-of-pd-states-using-ml

Discovery-of-PD-States-using-ML

Code to supplement the analysis presented in 'Discovery of Parkinson's disease states using a machine learning approach'. If you are would like to use the exact model or cohort presented in the paper, please reach out to us directly.

Data availability

The code in this repositiory leverages data made available via PPMI. For information about the dataset and to gain access, visit ppmi-info.org.

Organization of the code

The analysis is organized as follows

Data pre-processing

Several notebooks are used to pre-process the raw data from ppmi. Motor_Data_Processing.ipynb, Non_Motor_Data_Processing.ipynb, Demographics_Data_Processing.ipynb process those measures respectively. Full_Data_Processing.ipynb merges those sources and addresses date entry errors. LEDD_Data_Processing.ipynb processes the medication information using functionality provided in calc_ledd.py. Train_Test_Split.ipynb creates the train/test split used in analysis.

Contrastive latent variable model

The processed data is used as input to a contrastive latent variable model as demonstrated in Learn_and_Apply_cLVM.ipynb. The code for the model can be found here.

Personalized input-output hidden Markov model

The latent representation is used as input to the personalized input-output hidden Markov model in Learn_and_Apply_PIOHMM.ipynb. The code for the model can be found here The latent representation data from the cLVM must first be processed into a format the progression model can use as demonstrated in LD_Data_Processing.ipynb

Analysis of results

Manuscript_Figures.ipynb presents code the generates the primary figures of the manuscript.

discovery-of-pd-states-using-ml's Issues

Problems related to model loss function

I'm really sorry to disturb you again. Actually, I have already raised an issue in 'DiseaseProgressionModeling-HMM
', but I didn't get a reply. I wonder if you omitted it. Here is the original question:

Dear kseverso:
Hello, I am trying to reproduce the PD progress model, but have encountered some difficulties. I hope to get some help from you.

I followed the steps you preprocessed in "Discovery-of-PD-States-using-ML", processed the PPMI dataset (I made some changes due to the dataset changes), and then applied it to the PIOHMM. However, even if I set k=8 and hope to get 8 states, I only get 3-4 results in the training set, which are often in the time series t<5(T=31) I have already get the final state, for example, states 3 and 4. This situation becomes more and more significant with more iterations, and may even end up with only two states.
Besides, I may not know exactly what the model parameter learning steps mean. Are the ELBO and log_prob (self.ll) obtained at each iteration concepts similar to 'loss' in neural networks? According to my observation, ELBO was around 30,000 after the first iteration, then changed to around -10000 after the second iteration, and remained negative at -100000~-120000. Log_prob, on the other hand, keeps at about 110000, iterating at the learning rate of 1E-18, and fluctuates around 120000 or so when it reaches 20 times (convergence is impossible even using usE_CC convergence standard). I wonder how ELBO and log_prob change when you apply it into PPMI datasets? And what orders of magnitude are they?
Looking forward to your reply!

Data Set Changes

Dear Dr Kristen A Severson:
I am a current PhD students of Zhejiang Sci-Tech University in China. I major in "Biomedical Engineering". Recently, I found one of your articles, titled "Discovery of Parkinson’s disease states and disease progression modelling: a longitudinal data study using machine learning" in The Lancet Digital Health.When I run Demographics_ Data_ In Processing.ipynb, I did not find Randomization on the PPMI official website_ Table.csv dataset. I wonder if there are other datasets that can replace it.
Thank you very much for your kind consideration and I am looking forward to your early reply.
Sincerely: Xu Zeqi
My Email address is:[email protected]

Could i use the exact model or cohort presented in the paper?

Hi, kseverso
i'm a Computer Postgraduate ，i have read the paper and want to reproduce it, Could i got the exact model or cohort presented in the paper? If possible, how can I contact you?

Academic Communication

Dear Dr Kristen A Severson:
I am a current PhD students of Zhejiang University in China. I major in "Biomedical Engineering". Recently, I found one of your articles, titled "Discovery of Parkinson’s disease states and disease progression modelling: a longitudinal data study using machine learning" in The Lancet Digital Health. This is a very interesting and valuable project, I found it may help me achieve my goals in this research field. This would make a really positive contribution to my work. When I wants to use the methods of article to apply to my research, but I can't implement using the cLVM and PIOHMM model to achieve low-dimensional representation and discover latent state of disease, I am wondering if you could kindly send me the source code and the necessary information about it. I promise they will be used only for research purposed. Due to your mailbox is set to automatically return,you cannot receive my emails. So,I have chosen to leave you a message on github.
Thank you very much for your kind consideration and I am looking forward to your early reply.
Sincerely: Zhang Suixia
My Email address is: [email protected]