radinhamidi/cnn-denoiser

EE8204 Final Project

Image Denoising Using Variational Convolutional Neural Network

We propose a novel architecture for image reconstruction by utilizing convolutional neural networks (CNN). The main idea is to use such networks to learn statistical distribution of pixels and extract concepts in form of feature vectors. Most of the image noises are come from additive Gaussian Noise (AWGN). By changing ordinary CNN architecture and also using generative models these additive noise can be detected and deducted from the disrupted images. Using variational neurons as the fundamental elements of the proposed architecture will let us to denoise both low-level and high-level distorted images. Contrary to the existing discriminative denoisers, our proposed model will need significantly less amount of training data to achieve acceptable performance and also it needs less time for calculation. We ran various set of experiments on proposed and competitive models. This conducted evaluations show that our model is efficient and making it practical denoising applications.

Contributions

Our contributions can be enumerated as follows:

• We formally define problem of image denoising andproposed a novel approach based on convolutional neuralnetwork. This neural network will use variational autoen-coders setup to capture Gaussian distributions over thepixel values.
• We will test our model with wide range of noise levels tostudy robustness of model against different type of noises.
• We evaluate our proposed model over the well-knownimage dataset to compare the results. We will show thatproposed model will outperform current methods that arebased on other approaches

Files and Folders Guide

• ./feature_maps: contains the feature maps outputs.
• ./figures: Contains the figures during train/test of the program.
• ./gifs: Gifs for the readme file.
• ./logs: Training/Testing log files.
• ./models: model architecture files in json format generated by the Keras
• ./weights: model weights in hd5 format generated by the Keras
• stl10_input.py: Dataset downloading file.
• simple_autoencoder.py: Model builder Singe layer version
• stack_autoencoder.py: Model builder Stacked version
• classifier.py: Classifer for the outputs of the autoencoder for the comparison.
• util.py: useful functions that become handy in the software.
• KNN.py: A KNN core for the comparison.
• load_data.py: Data preparation for the models handled here. Resizing, compression, etc.

Dataset files

You can download the dataset by runing stl10_input.py file. After running the file, it automatically download the dataset and save it to the local directory. Also there are moudles in the file for single image plot and read labels. Local directories setting can be changed using these variables at the beginning of the file:

• Path to the directory with the data: DATA_DIR = './data'
• Url of the binary data: DATA_URL = 'http://ai.stanford.edu/~acoates/stl10/stl10_binary.tar.gz'
• Path to the binary train file with image data: DATA_PATH = './data/stl10_binary/train_X.bin'
• Path to the binary train file with labels: LABEL_PATH = './data/stl10_binary/train_y.bin'

Train a New Model

You can train a model by running the stacked_autoencoder.py or simple_autoencoder.py file. After training the mode, neural network architecture, weights and outputs are available in their folders. For the classification, you need to run the classifier.py file.

Results

Three type of outputs are saved during running the code:

1. Figures of the training/testing processes. Folder ./figures contains
2. Tained models Archs. Folder ./models contains model architecture files for transfer-learning or future evaluation
3. Tained models Weights. Folder ./weights contains model weights files for transfer-learning or future evaluation
4. Featured maps of images are also saved seperately so they can be used for other types of Machine Learning tasks. For example Clustering, Classification using other moethods like SVN. Folder ./feature_maps contains the file.

Demo

In the following you can see two sample of denoising process captured in different epoches. They are put in sequence as a gif for better infrerence.

Classification On Denoised Images

We can perform a classification on de-noised images in order to seeimpact of denoising on a classifier performance. Therefore a classifier trained to classifies the CIFAR-10 dataset without noises. Then a noisy version of CIFAR-10 is given to the de-noising model and results are used for the classification. The difference between original images classification results and reconstricted version of noisy ones can depict performance of the denoising model.

Evaluation

Evaluation results can be found below:

You can see that most of the confusion is caused when objects are visually similar to eachother. For example, Dog <=> Horse or Ship <=> Truck.

CIFAR-10	Original (Without Noise)	Reconstructed The Noisy Input
Accuracy	94.7%	88.26%

Preliminary Report

You can find prelimiary repot here. It's submitted for mid semester.

Acknowledgment

This project is done for EE8204 final project.
Instructor: Dr. Kandasamy Illanko
by Radin Hamidi Rad
Ryerson University
Summer 2020