hiuhiuwong/hdrl4cellmigration

Hierarchical deep reinforcement learning reveals novel mechanism of cell movement

Introduction

This is accompany code and data associated with the paper submission 'Hierarchical deep reinforcement learning reveals novel mechanism of cell movement'. And it was cloned and updated from https://github.com/zwang84/hdrl4cellmigration/

Package requirements

Python3 (tested on 3.6.2)
PyTorch 0.2 (tested on 0.2.0_3 (old) and 0.4.1) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., … Chintala, S. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32 (pp. 8024–8035). Curran Associates, Inc. Retrieved from http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf https://pytorch.org/ Mesa (tested on 0.8.1)
Kazil J., Masad D., Crooks A. (2020) Utilizing Python for Agent-Based Modeling: The Mesa Framework. In: Thomson R., Bisgin H., Dancy C., Hyder A., Hussain M. (eds) Social, Cultural, and Behavioral Modeling. SBP-BRiMS 2020. Lecture Notes in Computer Science, vol 12268. Springer, Cham. https://mesa.readthedocs.io/ PIL 4.2.1
Umesh, P. (2012). Image Processing in Python. CSI Communications, 23. https://pillow.readthedocs.io/ scikit-learn 0.19.1
Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825-2830, 2011. https://scikit-learn.org/ numpy (tested on 1.15.0)
Harris, C.R., Millman, K.J., van der Walt, S.J. et al. Array programming with NumPy. Nature 585, 357–362 (2020). DOI: 0.1038/s41586-020-2649-2. (Publisher link). https://numpy.org/

File structure

./data/: folder with textual data of nuclei
./data/data_description.txt: a brief description of the input textual embryonic data.
./data/Cpaaa_0: embryonic data for Cpaaa migration training and evaluation.
./data/Cpaaa_[1-3]: embryonic data for Cpaaa test case.
./data/mu_int_R_CANL_[1-2]: embryonic data for mu_int_R and CANL case.

./trained_models/:folder with all the pre-trained models.
./trained_models/hdrl_llmodel.pkl: checkpoint of the trained lower-level DQN
./trained_models/motion_model.pkl: checkpoint of the trained Motion Model
./trained_models/neighbor_model.pkl: checkpoint of the trained Neighbor Relationship Model
./trained_models/TMM.pkl.link: the download link for the TMM

./saved_data/: folder that used for saving the output data when exploring the successful scenarios. (see below)

./embryo.py: functions for parsing the embryo data
./draw_plane.py: visualization
./model.py: agent-based model for cell migration using Mesa. It uses the neighbor_model.pkl to determine neighbor relationship, and the motion_model.pkl to check the movement type
./model_obs_cpaaa.py: a cell migration environment that uses TMM to detect emerging features in the Cpaaa case.
./model_obs_mu.py: a cell migration environment that uses TMM to detect emerging features in the mu_int_R and CANL cases.
./run.py: an HDRL Model for cell migration using the model.py environment

Usage

1. Explore the successful scenarios: Command: python3 run.py
   Three Files are generated in the 'saved_data' folder after the evaluation:
   

   • movement_index.pkl: the movement index (1 for directional movement and 0 for random movement) of Cpaaa at each time step.
     
   • cpaaa_locations.pkl: location of Cpaaa at each time step.
     
   • target_locations.pkl: location of the target cell (ABarpaapp) at each time step. If ABarpaapp is not born, [0,0,0] is used as a placeholder.
     

2. Test the Cpaaa migration movement index in three different embryos:
   (1). Download the observational data and the TMM checekpoint in the google drive:
   observational data: https://drive.google.com/drive/folders/1_w0p7t_dmTha8ODTgosGXZO9gMRmYd3N?usp=sharing
   TMM checkpoint: https://drive.google.com/file/d/172FC8-8074mxotD8JSZemeRcA3v57ZF6/view?usp=sharing
   (2) Command: python3 model_obs_cpaaa.py --emb [1,2,3]
   Movment index will print out after the program is done. The results are used for Figure 5B,5C, and 5D
   

3. Test the movement index of the mu_int_R and CANL migration in two embroys:
   (1). Download the observational data in the google drive:
   observational data: https://drive.google.com/drive/folders/12JOhhz9LxvNig4BgcOidWTUqjrfW08t-?usp=sharing
   (2) Command: python3 model_obs_mu.py --cell [mu_int_r, canl] --emb [1,2]
   Movment index will print out after the program is done. The results are used for Figure 6C, and Figure 6D
   

Citation

Will update after paper submission.