This project is created for academic purpose and also available publicly. More information about the competition can be found here.
- Anaconda
- Kaggle API:
- If you are using window, you will have to use window's linux subsystem to run the bash script which download the dataset. Or you can download directly from kaggle.
- Clone this repository recursively (to download the submodules):
git clone --recursive https://github.com/rxng8/Chest-Xray-Abnormalities-Detection.git
Note: The object detection api is followed directly from Tensorflow Object Detection API's tutorial.
Note Currently, the detection api use COCO Dataset. Please make sure to download that dataset (from Kaggle), unzip and put it in the dataset directory. Or you can use the script in the root directory that downloads the dataset for you (Linux-like machine required, kaggle installed required):
bash download_dataset.sh coco
-
Installation according to this tutorial, briefly:
- Create conda virtual environment.
- Install tensorflow (If using GPU version, install CUDA toolkit, cuDNN).
- Download the Tensorflow Model Garden. (downloaded in this git repo)
- Install Protobuf.
- Install COCO API.
- Install the Object detection API from the Tensorflow Model Garden repo.
-
Training custom object detector.
- Prepare the workspace as mentioned in this tutorial.
- Export csv files for the current dataset:
# At the root directory python export_COCO_csv.py
- Copy all images from the
train_images
andval_images
directories in theCOCO Dataset
into thetrain
andtest
directories in the tensorflow workspace setup in the first bullet point. - Generate the
tfrecord
files for the dataset in order to call the training API:
# In the root directory. Generate tfrecord for training. python ./Tensorflow/scripts/generate_tf_records.py -j ./tmp/train.csv -i ./Tensorflow/workspace/training-demo/images/train -l ./Tensorflow/workspace/training-demo/annotations/label_map.pbtxt -o ./Tensorflow/workspace/training-demo/annotations/train.record # In the root directory. Generate tfrecord for validation. python ./Tensorflow/scripts/generate_tf_records.py -j ./tmp/val.csv -i ./Tensorflow/workspace/training-demo/images/test -l ./Tensorflow/workspace/training-demo/annotations/label_map.pbtxt -o ./Tensorflow/workspace/training-demo/annotations/val.record
- Download any of the pre-trained model in this model zoo (I used
SSD ResNet50 V1 FPN 640x640
model). Please refer to this part of the tutorial for more information. - Create directory and config the model according to this part of the tutorial.
- in the
- Copy the file
model_main_tf2.py
to the working directory and train the model by following the command, according to this part:
# cd inside training-demo directory. python model_main_tf2.py --model_dir=models/mymodel --pipeline_config_path=models/mymodel/pipeline.config
-
Evaluating the detector.
[To be editted]
- Visualizing data and training performance with Tensorboard.
[To be editted]
[To be editted]
- [3 hours] Created project pipeline template. More information can be found in issue #2!
- [6 hours] Research about R-CNN. Here is the list of articles about object detection (R-CNN techniques) I have read and gain an insight to it:
- [6 hours] R-CNN, Fast R-CNN, Faster R-CNN, YOLO โ Object Detection Algorithms:
- [1.5 hours] R-CNN: This paper is the original technique where the images is segmented into more than 1000 bounding boxes and then classified by the network.
- [1.5 hours] Fast R-CNN: In this paper, instead of proposing 1000 bounding boxes and feed the region proposal to the convolutional neural network as the original R-CNN technique, the image is passed through a convolutional neural network to predict the RoI feature vector. Then the RoI vectors is passed through the fully connected network.
- [1.5 hours] Faster R-CNN: This paper improves the Fast R-CNN by upgrading the region proposal process. Instead of using search algorithm to propse regions, this paper has a separate network to predict the region of the object, then the region is passed to the RoI pooling layer as usual.
- [1.5 hours] YOLO paper: This algorithm is different from above algorithms by dividing the whole image into grids of probability. Then it uses a convolutional neural networks to predict each group of bounding boxes to be the bounding boxes of the object.
- After reading those articles, I come to the conclusion that for the Chest X-ray data, there are a lot of small-scale object/abnormalities, so the large frame/grid in YOLO algorithm will have a hard time predicting. Therefore I choose to follow the R-CNN path.
- [6 hours] R-CNN, Fast R-CNN, Faster R-CNN, YOLO โ Object Detection Algorithms:
- [5 hours] Implement data preprocessing pipeline, in particular:
data
: Contains data preprocessing utilities and dataset models.
- [4 hours] Implements the R-CNN neural network with high-level library, in particular:
models
: Contains model building utilities, and initial code of Faster R-CNN model.
API
: Integrate the use ofTensorflow Object Detection API
.- [3 hours] Install the tensorflow model library and setup protobuf.
- more reference
- [2 hours] Write COCO Dataset Wrapper for further processing, especially export to
tfrecord
. - [2 hours] Generate image annotation and
tfrecord
annotation for tensorflow model usage.
- [7 hours] Successfully implement the data pipeline and export to tfrecord.
- [4 hours] Implement the data wrapper class
DatasetCOCO
. Sucessfully generate csv file as needed. - [3 hours] Copy and re-implement the scripts
generate_tf_records.py
. This file takes inputs csv containing paths, bounding boxes, classes, etc, to generatetfrecord
files for api training. (I spend almost 2 hours on fixing the bugs: sanity error)
- [4 hours] Implement the data wrapper class
- [2 hours] Writing documentation for methods, installation, and requirements.
- [2 hours] Train and export models to be evaluated in future.
- (Do not counted this) [26 hours] (Have not done yet! My GPU is too small) Wait for the model to be trained on GPU.
- [2 hours] Writing exporting models and evaluate on the validation set. (To be added!)
- [4 hours] Fixing environment bug.
- [5 hours] Reading and preparing
DatasetCOCOPytorch
fordataset.py
for pytorch transfer leanring and prepare training pipeline. Here is the architecture for the Faster R-CNN: Refer to this link for further reading. Here is the kaggle notebook that use transfer learning. Here is the code block that replace the top layer with our final classification layer:model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True) num_classes = 15 # 14 Classes + 1 background # get number of input features for the classifier in_features = model.roi_heads.box_predictor.cls_score.in_features # replace the pre-trained head with a new one model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
- [1 hours] Research about transfer learning and pytorch custom dataset.
- [2 hours] Walking-through and writing Pytorch dataset:
DatasetCOCOPytorch
indataset.py
for pytorch transfer leanring. - [2 hours] Wrting pipeline: training pipeline.
- [3 hours] Finish the Pytorch dataset:
DatasetCOCOPytorch
indataset.py
for pytorch transfer leanring.__len__()
: Get the length of the dataset.__getitem__()
: Get current (image, label) at the current index in the dataset.
- [6 hours]: Create conda environment and version-controlled libraries. We can now run the conda virtual environment with file
environment.yml
- Now we can create virtual environment with the following code:
conda env create -f environment.yml
- Creating environment costs so much time! Prepare libraries:
conda install -n pytorch matplotlib pandas albumentations opencv -c conda-forge conda install -n pytorch pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge conda install -n pytorch tensorflow-gpu -c anaconda
- [9 hours] Finish the whole pipeline and train models Faster R-CNN with transfer learning with pytorch. Here is the kaggle colab notebook
- [2 hours] Doing exam. (Private code!)
- [2 hours] Creating new version of the notebook which is more clear. (
faster_rcnn2.py
) - [1 hours] Produce and submit the first submission from the
faster_rcnn2.py
notebook (0.087 point). - [6 hours] Recreate, redesign
faster_rcnn.py
and successfully trained with transfer learning. Here is the result: Link to the pytorch saved model - [1 hours] Writing evaluation and building submission.