wurenkai/mcf-smsis

MCF-SMSIS:Multi-tasking with complementary functions for stereo matching and surgical instrument segmentation

Renkai Wu, Changyu He, Pengchen Liang, Yinghao Liu, Yiqi Huang, Weiping Liu, Biao Shu, Panlong Xu, Qing Chang*

1. Shanghai University, Shanghai, China
2. Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
3. Imperial College London, London, United Kingdom
4. Shanghai Microport Medbot (Group) Co.,Ltd., Shanghai, China

Demo.mp4

The manuscript is under review. We are providing validation codes to facilitate the review. We will continue to update our code and README.md.

How to run the code

Environment

• NVIDIA GeForce RTX 4080 Laptop GPU (12GB)
• Python 3.8
• Pytorch 1.12

Install

Create a virtual environment and activate it.

conda create -n MCF python=3.8
conda activate MCF

Dependencies

conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch -c nvidia
pip install opencv-python
pip install scikit-image
pip install tensorboard
pip install matplotlib 
pip install tqdm
pip install timm==0.5.4

Data Preparation

The SCARED dataset should be obtained from the official website under a confidentiality agreement. The dataset also needs to be corrected and the correction toolkit is available at scared-toolkit.

Model weights for each stage

• S1-d1 1.Google drive 2.Baidu drive Link Password:89ja
• S2-d1d2 1.Google drive 2.Baidu drive Link Password:nv68
• S2-d1s 1.Google drive 2.Baidu drive Link Password:ttr4
• S3-d1d2s 1.Google drive 2.Baidu drive Link Password:9tn7
• S3-d1sd2 1.Google drive 2.Baidu drive Link Password:z4b4
• S1-s 1.Google drive 2.Baidu drive Link Password:nwuj
• S2-sd1 1.Google drive 2.Baidu drive Link Password:vbrg
• S3-sd1d2 1.Google drive 2.Baidu drive Link Password:vtlp

Validation and Testing

• You can proceed to run the test_seg_depth.py and test_seg.py files to verify the parallax and segmentation performance of the model, respectively.
• The test_only_depth.py file is needed to verify the model performance without combining the features in the segmentation and decoding part.
• To output the parallax result plot, you can run the test_save_disp.py file and save the result to the test folder.
• To output the segmentation results, run test_save_seg.py and save the results to the results folder.

Acknowledgements

Thank you for the help provided by these outstanding efforts.

• MSDESIS:
• CGI-Stereo: