standwisdom/pca-ggnet

A radiologist-like agent based on reinforcement learning to predict case-level ISUP grade in prostate cancer for restriction of upgrading and downgrading risk of pathological evaluation

PCa-GGNet

PCa-GGNet: A radiologist-like framework for interpretable prediction of prostate grade groups to restrict risk of upgrade and downgrade in needle biopsy

The goal of this repository is:

• to help researchers to reproduce the PCa-ISUPNet framework and expand for other prostate research or relevant research.
• to help researchers to build a generator-net alone for predicting ISUP grade of radical prostatectomy in case-level (multi-category),
• to help researchers to build an action-net alone for attentional slice searching.

Installation

1. python3 with anaconda
2. pytorch with/out CUDA
3. pip install pretrainedmodels
4. pip install pandas

Prepare your data for generator-net:

The training list was essential for the Framework.

For generator-net, you should split your 3D MRI into 2D slice, and record the data-ID, label, flag of tumor slice, and path of every slice as .csv or .xlsx. The names of columns were 'ID','label','Z', and 'path' in our project. More details can be found in DataSet.py and 'generatorNet_trainlist_example.xlsx'.

For action-net, you should first extract the CNN features of each slice (not only tumor slice) by generator-net. Then, you should record the data-ID, label, flag of tumor slice, predicted result of the slice with probability, and extracted features as .csv or .xlsx. More details can be found in action-net.py and actionNet_trainlist_example.xlsx.

Training for generator-net

python generator-net.py

Features extraction by generator-net

python generator-net-forward.py

Training for action-net and results of case-level prediction

python action-net.py

Process of our method

Introduction of PCa-GGNet

PCa-GGNet framework was proposed by three steps. Two basic units were prepared during the training phase, which were generator-net and action-net. A classifier based on tumor slice of T2WI was established for five-category prediction at slice-level (i.e generator-net). Next, action-net was training for attentional slice searching using features and classification results, which were generated from the generator-net. The generator-net and action-net were build step by step. During the prediction phase, three steps were required to predict case-level GG by the PCa-GGNet framework. In the first step, the middle slice was selected as input to generator-net, and CNN features and prediction based the slice was generated from the generator-net. In the second step, CNN features were employed for action-net and produced action order based on action rules. Last but not final, a checkpoint was set to draw the conclusion based on the action order from action-net. If the framework running circle was not satisfied with the condition of the checkpoint, which was experienced early stop or action of stay in place, the framework will update current attentional slice and repeated the process of step1 and step2. Otherwise, the case-level prediction would adopt a result of the attentional slice generated from the last circle.

Materials for training

• For CNN-based generator-net, tumor slices of T2WI were used as inputs, and the label of each slics was the patient-level pathological evaluation Grade Group of radical prostatectomy. This was a process of weakly surprised leaning.
• For DRL-based action-net, features of all slices, including normal slice, was extracted by a trained generator-net. Training environment for action-net was generated by CNN-based features, slice-level predicitons. More details was introduced in the source code by commentary.

Acknowledgement

Thanks to the https://github.com/Cadene/pretrained-models.pytorch for pretrained ConvNets with a unique interface/API inspired by torchvision.
Thanks to the https://github.com/sweetice/Deep-reinforcement-learning-with-pytorch for implemention of basical DQN framwork in pytorch.
Thanks to Peking University Third Hospital (PUTH) and Peking University People Hospital (PUPH) for data support.
Thanks to Key laboratory of molecular imaging, Insistute of automation, Chinese Sciences of Academy for support of platform.
Thanks to LIST, Key Laboratory of Computer Network and Information Integration, Southest University for technical support.

Reference

Shao, L., et al., Radiologist-like artificial intelligence for grade group prediction of radical prostatectomy for reducing upgrading and downgrading from biopsy. Theranostics, 2020. 10(22): p. 10200-10212.