thtang/chexnet-with-localization

Weakly Supervised Learning for Findings Detection in Medical Images

CheXNet-with-localization

ADLxMLDS 2017 fall final

Team:XD

黃晴 (R06922014), 王思傑 (R06922019), 曹爗文 (R06922022), 傅敏桓 (R06922030), 湯忠憲 (R06946003)

Weakly supervised localization :

In this task, we have to plot bounding boxes for each disease finding in a single chest X-ray without goundtruth (X, Y, width, height) in training set. The workflow is shown below:

Workflow :

1) Predict findings 2) Use the classifier to plot heatmap (Grad-CAM) 3) Plot the bounding box base on Grad-CAM ### Package : `Pytorch==0.2.0`   `torchvision==0.2.0`  ` matplotlib`  ` scikit-image==0.13.1`  ` opencv_python==3.4.0.12`  ` numpy==1.13.3`  `matplotlib==2.1.1`  `scipy==1.0.0`   `sklearn==0.19.1`  

Environment:

• OS: Linux
• Python 3.5
• GPU: 1080 Ti
• CPU: Xeon(R) E5-2667 v4
• RAM: 500 GB

Experiments process:

1. preprocessing:

python3 preprocessing.py [path of images folder] [path to data_entry] [path to bbox_list_path] [path to train_txt] [path to valid_txt] [path of preprocessed output (folder)]

2. training:

python3 train.py [path of preprocessed output (folder)]

3. local testing:

python3 denseNet_localization.py [path to test.txt] [path of images folder]

4. Output txt format:
   After running denseNet_localization.py, you would get a txt file. The format is shown below:
   

[image_path] [number_of_detection]
[disease] [x] [y] [width] [height]
[disease] [x] [y] [width] [height]
...
[image_path] [number_of_detection]
[disease] [x] [y] [width] [height]
[disease] [x] [y] [width] [height]
...

---

For DeepQ platform testing:

upload deepQ_25.zip to the platform. Then use following command:

python3 inference.py

---

5. For visualization, please refers to issue. Credit to Sadam1195.

Note :

In our .py script, I used the following script to assign the task running on GPU 0.

import os
os.environ['CUDA_VISIBLE_DEVICES'] = "0"

Model :

* Image is modified from Ref [2].

Result :

Prediction

Heatmap per disease Visualization of some heat maps with its ground-truth label (red) and its prediction (blue) selected from each disease class. (From top-left to bottom: Atelectasis, Cardiomegaly, Effusion, Infiltration, Mass, Nodule, Pneumonia and Pneumothorax)

Bounding Box per patient Visualization of some images with its ground-truth label (red) and its prediction (blue) selected from each disease class.

Refers to the report for more experiment results.

Reference:

1. ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases [Arxiv]
2. LEARNING TO DIAGNOSE FROM SCRATCH BY EXPLOITING DEPENDENCIES AMONG LABELS [Arxiv]
3. CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning [Arxiv]
4. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization [Arxiv]

Contact:

Feel free to contact me ([email protected]) if you have any problem.

chexnet-with-localization's Issues

Localization Eval Metric Definition&&Code Wanted.

Recently I am working on reconstruct your work's localization part.
I fill up the Eval Part with Precision and Recall as eval metric.
But the I just can not get your report's result even with the boundingbox.txt you provided.
So I wonder if you could provide me with the Eval related code.
I promise for scientific research only.

How to run find_bbox_size.py?

Thank you very much for your project！
I am trying to run the find_bbox_size.py for obtaining IOU scores.

This is the annotation in the find_bbox_size.py.

Find the size of bounding boxes
Usage: python3 find_bbox_size.py <train_heatmap> <valid_heatmap>
<{train,valid}heatmap> contains .npy files named 'heatmap_.npy'
describing the heatmap of positive cases predicted by our model
files in <valid_heatmap> will only be used to calculate IOU scores
is the order of each validation image (in valid.txt)
is the disease class in alphabetical order (0-7)

Where to obtain <train_heatmap> <valid_heatmap>?
Follow your prompts，<{train,valid}heatmap> contains .npy files named 'heatmap_.npy'

I really hope to receive your answer！
Thanks！

Error in During testing the model need the number of output to perform testing?

I am running test file to validate the results.
The data loaded and preprocess sucessfully and when I load model with out_size as given in code to 8 its says the below erorr?
What is the solution?
Did the code need to update?
or model in the Pytorch are missing something.

File "C:\ProgramData\Anaconda3\envs\chest-loc\lib\site-packages\torchvision\models\densenet.py", line 98, in init layer = _DenseLayer(num_input_features + i * growth_rate, growth_rate, bn_size, drop_rate) File "C:\ProgramData\Anaconda3\envs\chest-loc\lib\site-packages\torchvision\models\densenet.py", line 77, in init self.add_module('norm.1', nn.BatchNorm2d(num_input_features)), File "C:\ProgramData\Anaconda3\envs\chest-loc\lib\site-packages\torch\nn\modules\module.py", line 178, in add_module raise KeyError("module name can't contain "."") KeyError: 'module name can't contain "."'
Help me in this regard.
Thanks
@thtang

How to do inference using this model?

I don't have enough RAM to train this model again. Is there any way to use this model to do inference on my data?

how to visualise the bounding_box.txt file in an image?

I am able to generate the bound_box.txt file after I execute the denseNet_localization.py file.

how to visualize the bound box text file into the input image? as you mentioned in ground-truth label (red) and its prediction (blue).

Thanks in advance..

list out of index issue arise in preprocessing.py

Traceback (most recent call last):
File "/content/CheXNet-with-localization/preprocessing.py", line 11, in
image_folder_path = sys.argv[1] # folder contain all images
IndexError: list index out of range
any sloution??

Weights not used in training batch

Hi! Thanks for this project. I noticed the weights for each training batch are not used. How did you incorporate them in training? I read in your report that your mean AUC ended up being better with them.

Invalid layer name: module.densenet121.features.denseblock4.denselayer16.conv.2

Hi,

I got the below error while running denseNet_localization.py file.

Traceback (most recent call last):
  File "/media/administrator/D/XRay/CheXnet_Demo/denseNet_localization.py", line 227, in <module>
    output = gcam.generate(target_layer="module.densenet121.features.denseblock4.denselayer16.conv.2")
  File "/media/administrator/D/XRay/CheXnet_Demo/denseNet_localization.py", line 186, in generate
    fmaps = self._find(self.all_fmaps, target_layer)
  File "/media/administrator/D/XRay/CheXnet_Demo/denseNet_localization.py", line 174, in _find
    raise ValueError('Invalid layer name: {}'.format(target_layer))
ValueError: Invalid layer name: module.densenet121.features.denseblock4.denselayer16.conv.2

any help would be really appriciated.. Thanks in advance..

avg_size array in denseNet_Localization.py

What does the avg_size array in the bounding box procedure represent, and how did you retrieve it? Thanks!

ValueError: Target and input must have the same number of elements

calculating loss between tensors that don't have the same size of elements
outputs & labels_sub
[64,8] & [64,7]

ValueError: Target and input must have the same number of elements. target nelement (448) != input nelement (512)

CheXNet-with-localization/train.py

Line 170 in a5c0c8b

loss = criterion(outputs, labels_sub)

issue

Traceback (most recent call last):
File "/content/CheXNet-with-localization/preprocessing.py", line 11, in
image_folder_path = sys.argv[1] # folder contain all images
IndexError: list index out of range

Hello, What license is your project?

Heatmap Visualization of

Could anyone visualize the heatmap using GradCAM?

After changing the name in denseNet_localization.py module.densenet121.features.denseblock4.denselayer16.conv.2 to module.densenet121.features.denseblock4.denselayer16.conv2

It then returns Nan values for the heatmap.

the format of the input data

Could you provide with the format or demo of the input data?
data_entry.csv bbox_list_path.csv

How to remove the memory problem?

Hello, sir thanks for the repo. I have done image preprocessing. Now I want to train the network but as the memory dumped in preprocessing but still, I have been done it. In training, the system hanged and RAM memory is full, I have 64GB ram with 50GB+ paging (virtual ram).
Do you have any idea how to tackle this error?
@thtang Thanks for your concern

loading non existent file 'train_X_small.npy'

the file 'train_X_small.npy' is loaded from [path of preprocessed output (folder)], but it was never created.
should it be downloaded from somewhere?

CheXNet-with-localization/train.py

Line 38 in 72a91f1

self.X = np.uint8(np.load(data_path + "train_X_small.npy")*255*255)

ModuleNotFoundError: No module named 'judger_medical'

I trying run Inference,py file from deepQ_25.zip and getting
ERROR
ModuleNotFoundError: No module named 'judger_medical

thanks.

How to run this on a single image ?

I want to use this model and run it on a single image say 'xray.jpeg'. What needs to be done ?

where's thresholds.npy come from?

Hi, i found you using different thresholds for each class, this seems interesting.
How did you generate the thresholds for each class? and why?

About BBox_List_2017.csv

Hello, I want to train my own data using your weakly supervised net, however, while preparing the necessary files, I'm in trouble of creating my own BBox_List_2017.csv ,cause I have about 2000 labels to calculate these four values : x y w h, my thought on this at present is that maybe I could read the xml files and automatically calculate those four values, then put them in the csv file. But It's so hard to find the algorithm to complete this requirement.

I'm wondering what is your measure to finish the BBox_List_2017.csv. Could you please give me some advice on this? I would appreciate that a lot!