brian-mchugh/cnn-classification

Using Python and Pillow, create and organize a dataset of images of common household items in various positions. Utilize Matplotlib, TensorFlow, Keras and three CNN models pre-trained on the ImageNet dataset to classify the objects. Review model performance with Pandas and Tableau and build a website to visualize the Tableau tables and charts.

CNN Classification Characteristics

Project Goal

It's known that image recognition systems are easily fooled by well-known objects that appear in unnatural positions, e.g., a school bus on its side: Google's image recognition AI fooled by new tricks.

So we wonder what characteristics are having more of an impact on CNNs failing to correctly classify an object in an image. We will use a model (ResNet, Inception) pre-trained on the ImageNet dataset and then introduce images of common objects in a variety of situations to see which are most problematic for the models.

Proposed scenarios:

• standard objects in standard postions but with the whole image rotated 90, 180, 270 degrees prior to being fed to the model

• standard objects in unnatural postions (chair lying on its back on the floor)

• standard objects in unnatural postions but rotated 90, 180, 270 degrees to make their position appear natural (teapot lying upside down on a countertop, with the image being rotated 180 degrees so that it appears upright)

• standard objects in unnatural locations (an upright chair facing a bed or a toilet)

We will also attempt to implement LIME to see if we can't ascertain what the models are having the most trouble with when failing to identify the objects.

Tools

We will utilize Keras and TensorFlow when working with the CNNs and we will use Matplotlib to display our findings.