deepwise-code/dlia

DLIA

create time: 2020.08.26

Introduction

This repository contains the source code and the trained model for our work Deep Learning for Intracranial Aneurysm Detection in Computed Tomography Angiography Images.

Prerequisites

• Ubuntu: 16.04 lts
• Python 3.6.5
• Pytorch 1.0.1.post2
• NVIDIA GPU + CUDA_10.0 CuDNN_7.5

This repository has been tested on NVIDIA RTX2080Ti. Configurations (e.g batch size, image patch size) may need to be changed on different platforms.

Installation

Other packages are as follows:

• yacs
• nibabel
• scipy
• joblib
• opencv-python
• SimpleITK
• scikit-image
• numpy

Install dependencies:

pip install -r requirements.txt

Usage

We mainly use main.py (under the project) to train/val/test our models.

The following is one example:

python main.py --gpu 0 1 2 3  --train --config tasks/configs/aneurysm_seg.daresunet.yaml

The main parameters are following:

• --train: used to train the model.
• --test: used to test(val) the model.
• --config: the path to the configuration file(*.yaml).
• --resume(optional): the path to the checkpoint pth(resume the model).
• --gpu(default 0): decide to which gpu to select. Format: one or multiple integers(separated by space keys), such as
• --gpu 0 1 2 3
• --check_point(optional): the path to save the trained model, we usually specify the parameter in the config file, if you specify this parameter here, it will override this parameter in the config file.

Train

Run command as below.

python main.py --gpu 0 1 2 3  --train --config tasks/configs/aneurysm_seg.daresunet.yaml

Inference

the weight of the model: raws/weight/da_resunet.pth.tar

configuration file: tasks/configs/aneurysm_seg.daresunet.yaml

• TEST.DATA.NII_FOLDER: directory of input files
• TEST.DATA.TEST_FILE: list of file names
• TEST.SAVE_DIR: the directory to save results If you want to use your custom data, you need modify the yaml file to set the path and file names of the test data.

Run command as below.

python main.py --gpu 0 1 2 3  --test --config tasks/configs/aneurysm_seg.daresunet.yaml --check_point raws/weight/da_resunet.pth.tar

qucik start

you can quick start inference and visualization using vis_demo.ipynb

• the example image data: raws/image/example.nii.gz
• the ground truth : raws/image/example_mask.nii.gz
• the expected output: raws/example_seg.nii.gz

other

in the ./raws directory, we supply an example dataset

raws
├── image  # directory of image files
├── mask   # directory of ground truth
├── lesion_bbox.txt  # lesion bbox info 
├── part_train.txt  # train list
├── part_val.txt    # validation list
├── part_test.txt   # test list 

dlia's Issues

where is vis_demo.ipynb located?

I can not find this file in the repo.

bbox info

Hi, I wonder what are those numbers stand for in lesion_bbox.txt?

Request for Dataset

Hi! Could someone please help me obtain the data used to train this architecture?

example image inference is not working on test

I tried to run "test" on the example image you provided and used the weights you provided. The predicted seg is all blank (black) even though it has aneurysm. Any suggestion? is there any better data/image that can be used to test the model? How do we know the model is working?

Why the precision of the example so low?

I ran the code "python main.py --gpu 0 --train --config tasks/configs/aneurysm_seg.daresunet.yaml "
then train the example and val it
But the metrics always only
"lesion-level: precision=0.0001(1/6817),recall=1.0000(1/1)
person-level: Sensitivity=1.0000(1/1),Specificity=0.0000(0/0)"
Why so low ? Is that example right?
Or any wrong i used the code ?

How to handle image that is less than 80 slices?

I am testing on a MRI image sets, and some of the images have less than 80 slices, and the code throws the following errors during training, any advice how to handle images with < 80 slices?

concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/concurrent/futures/process.py", line 175, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 49, in sampe_subjects
img, gt, _delayed_transform_add_coords = AneurysmSampler(subject, aneurysm_bbox[subject], spacing_zxy, image, aneurysm).sample(patch_num)
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 296, in sample
negative_coords_list.extend(self.sample_negative_random(neg_num - neg_num // 2))
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 222, in sample_negative_random
raise e
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 215, in sample_negative_random
x = random.randint(0, self.img_size[0] - self.patch_size[0])
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/random.py", line 221, in randint
return self.randrange(a, b+1)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/random.py", line 199, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,-15, -15)
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "main.py", line 75, in
main()
File "main.py", line 71, in main
main_worker(args)
File "/home/ubuntu/DLIA/tasks/main.py", line 85, in main_worker
trainer.train()
File "/home/ubuntu/DLIA/tasks/main.py", line 40, in train
self.task.train(epoch)
File "/home/ubuntu/DLIA/utils/tools/util.py", line 47, in int_time
res = func(*args, **kwargs)
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_seg.py", line 61, in train
train_set = self.data_sampler.get_data_loader()
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 121, in get_data_loader
img, gt, _delayed_transform_add_coords = obj.result()
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/concurrent/futures/process.py", line 175, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 49, in sampe_subjects
img, gt, _delayed_transform_add_coords = AneurysmSampler(subject, aneurysm_bbox[subject], spacing_zxy, image, aneurysm).sample(patch_num)
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 296, in sample
negative_coords_list.extend(self.sample_negative_random(neg_num - neg_num // 2))
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 222, in sample_negative_random
raise e
File "/home/ubuntu/DLIA/tasks/aneurysm/aneurysm_loader.py", line 215, in sample_negative_random
x = random.randint(0, self.img_size[0] - self.patch_size[0])
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/random.py", line 221, in randint
return self.randrange(a, b+1)
File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/random.py", line 199, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,-15, -15)

Help Understanding How Patient-Level Metrics Are Computed

Hi, I'm a little confused how patient-level metrics are computed.

To be clear, on the following line, the variable person_negative_total is the denominator in calculating specificity, meaning that person_negative_total is true negative + false positive, which means that person_negative_total is the number of ground-truth negative patients.

DLIA/tasks/aneurysm/batch_metric.py

Line 212 in f85c5d8

specificity = person_negative_tp / (person_negative_total + 1e-6)

Is that correct?

If so, on the following line, person_negative_total is only incremented when _lesion_recall_total is 0. However, _lesion_recall_total is 0 when the number of true positives for lesions is 0, which is weird because this excludes the possibility that the model made in incorrect positive prediction i.e. _lesion_recall_total should also be incremented when the model predicts a false positive.

DLIA/tasks/aneurysm/batch_metric.py

Line 194 in f85c5d8

if _lesion_recall_total == 0: # for person negative case

Is the code wrong, or am I misunderstanding?

CPU RAM usage keeps growing during evaluation

I am training the model using gpu, with batch size = 16, and everything is fine.
The cpu memory keeps growing during evaluation. How to fix it?
I've tried torch.no_grad() and torch.cuda.empty_cache() , but they don't work.

model checkpoint

Hello,

would you please upload the model checkpoint that can be used for testing?

Best
Wendy

Where is code for normalization preprocessing?

The Nature Communications paper has a paragraph that discusses brightness normalization preprocessing. I'm having trouble finding where in the code this is performed. These two lines suggest to me that only one window interval is used, not three:

DLIA/tasks/aneurysm/datasets/aneurysm_dataset.py

Lines 30 to 31 in f85c5d8

	WL, WW = cfg.TRAIN.DATA.WL_WW
	img = set_window_wl_ww(img, WL, WW)

Can you please point me in the right direction?

I'm referring to the following paragraph in the paper:

"In the inference stage (Fig. 1b), the segmentation prediction of the whole volume was generated by merging the prediction of uniformly sampled patches. Two adjacent patches may have 1/8 overlap, in other words, the strides along the three axes were all 40. For each voxel, we used the highest probability from all enclosing patches as its final prediction. In order to detect IAs in some low-contrast images clipped by the default window interval of [0, 900], another two intervals of [0, 450] and [−50, 650] were used to normalize the source images. The setting was automatically selected according to the brightness distribution. Given a bone-removal CTA image, a threshold value such as 150 Hu was used to find the initial area of vessels and then the maximum connectivity area was kept as the final region of vessels. Histogram of the brightness of voxels in the region was analyzed to find suitable clipping interval. We counted the distribution of three intervals including [0, 200], [200, 300], and [300, 500], which corresponded to clipping intervals of [0, 450], [−50, 650], and [0, 900], respectively. Finally, the clipping interval corresponding to the dominated distribution interval was selected to normalize the source images."

Thank you for your help!

get my lesion_bbox.txt

If I have the mask how can I get my lesion_bbox.txt?

ValueError: empty range for randrange() (0, -15, -15)

Hi,

I am using my own dataset tyring to train the model to see how it performs.

The training process ran well for few epochs at the beginning, but I encounter an error during the 16th epoch:

Seems like something went wrong when loading the data.

Do you have any idea of the possible culprits?
Thanks in advance!

Yashi

How to solve the high FPs ?

lesion-level: precision=0.1570(122/777),recall=0.9444(119/126)
person-level: Sensitivity=0.9706(99/102),Specificity=0.2000(10/50)

Amount of GPU memory necessary to run the code.

Hello!

I am trying to run this code but I am stuck in error:

"RuntimeError: CUDA out of memory. Tried to allocate 500.00 MiB (GPU 0; 6.00 GiB total capacity; 3.07 GiB already allocated; 90.04 MiB free; 3.98 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting ma
x_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"

I tried to run it in an AWS server with 16 GB GPU and it gave me the same error.

What is the amount of GPU memory necessary to run this code? Could it be some issue while installing the dependencies?

Subject id

In the lesion_bbox.txt, is the first integer "1005642077" the Subject Id? How does it map to image file example_nii.gzip?

1005642077 7 245 292 10 14 12

Request for data

Can I get the images/data used to train the model?

How was Test / Inference Threshold of 0.3 Chosen?

In the aneurysm segmentation test function, there's a threshold value of 0.3:

DLIA/tasks/aneurysm/aneurysm_seg.py

Line 208 in f85c5d8

seg = (seg >= 0.30).cpu().numpy().astype(np.uint8) # binary, mask

How was this value chosen?

What is the difference between CaseTPLesionDICE and LesionRecallDice ?