surya-veer/realtime-digitrecognition

RealTime DigitRecognition using Convolutional Neural Network(CNN) with keras.

RealTime-DigitRecognition

RealTime DigitRecognition using keras/SVC and pygame.Project Report

Overview

Recently Deep Convolutional Neural Networks (CNNs) becomes one of the most appealing approaches and has been a crucial factor in the variety of recent success and challenging machine learning applications such as object detection, and face recognition. Therefore, CNNs is considered our main model for our challenging tasks of image classification. Specifically, it is used for is one of high research and business transactions. Handwriting digit recognition application is used in different tasks of our real-life time purposes. Precisely, it is used in vehicle number plate detection, banks for reading checks, post offices for sorting letter, and many other related tasks.

Description

This is a RealTime-DigitRecognition application which can predict output corresponding to handwritten images. I used SVC(support vector classifier) and sequential model of Keras for creating this predictive model. I trained SVC for 8X8 MNIST dataset, but the accuracy of this model is not good when I run this model on my handwritten images(600X600). It is due to resizing images from 600X600 to 8X8.It is important to get good results so I created a sequential model in keras and traied it on 28X28 MNIST dataset. Now it gives very good result on handwritten digits.

The interface is created by using Pygame. The image preprocessing is the most important in this project which I have done by using Scipy and OpenCV.

Dataset

MNIST is a widely used dataset for the hand-written digit classification task. It consists of 70,000 labelled 28x28 pixel grayscale images of hand-written digits. The dataset is split into 60,000 training images and 10,000 test images. There are 10 classes (one for each of the 10 digits). The task at hand is to train a model using the 60,000 training images and subsequently test its classification accuracy on the 10,000 test images.

Sample Images:

These are some sample images of the handwritten character from mnist dataset.

Dependencies

This is the list of dependencies for running this application.

• Skleran
• Keras
• tensorflow/theano
• Opencv
• Pygame
• Pandas
• Numpy
• Scipy
• Matplotlib

How to use

1. Download or clone this repository.

2. Extract to some location.

3. First, run app.py from RealTime-DigitRecognition folder.
   Now, Pygame window will open. It will look like this.
   
   
   
   

4. Draw the digits on left side of the window and output will appear on right side of the window.

5. Mouse handling:
   The right button is for resetting screen.
   The left button is for drawing.

Multi digit reconition

I am developing an efficient model for detection multiple digits on a single frame like number plate, phone number, cheque number etc.
Here are some results:

Demo

Please commit for any changes or bugs :)

realtime-digitrecognition's Issues

Error

after terminate to track any symbol I receive this:

File "D:\Project_Python\RealtimeDigitRecon\RealTime-DigitRecognition-master\RealTime-DigitRecognition-master\process_image.py", line 59, in get_output_image
im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
ValueError: not enough values to unpack (expected 3, got 2)

Python 3.7.1
Keras 2.0.8
Pygame 1.9.6

Best regards

Licence

Hi surya-veer!

Is there any Licence?

I want to repackage for education purpose...

Thanks

Using webcam to capture classifier source?

Hello!

Awesome work, I will try to study it very thougrougly!

I was just thinking if you have an example of using a webcam to be the image source of classifer? Where you can maybe take a picture of something you want to have the model classify?

add image that contain some digit on paper

i had a problem, maybe you can change how to change the input from realtime handwriting on pygame to image that contain some digit please. below i attach example of image that contain some digit on paper.

Specify the correct versions of used packages or update the code

Hi! I've tried to run your code, and there were 3 errors that I had to fix before it worked successfully.

1. Error: File "anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 325, in class_and_config_for_serialized_keras_object
   for key, item in cls_config.items():
   AttributeError: 'list' object has no attribute 'items'
   Please, specify the correct version of TensorFlow. For me, it didn't work with TF version 2.2.0 but successfully worked with TF version 1.15.0
   Solution: use TF 1.x

2. Error: File "RealTime-DigitRecognition/process_image.py", line 6, in
   from keras.models import load_model
   ModuleNotFoundError: No module named 'keras'
   Solution: modify file process_image.py line 6 to from tensorflow.keras.models import load_model

3. Error: File "RealTime-DigitRecognition/process_image.py", line 59, in get_output_image
   im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
   ValueError: not enough values to unpack (expected 3, got 2)
   This is from the new version of OpenCV. I use cv2 version 4.3.0, and cv2.findContours returns 2 arguments instead of 3. Probably specifying the correct version for OpenCV would also be useful.
   Solution: modify line 59 in the file process_image.py to contours,hierarchy = cv2.findContours(thresh, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE).

Great utility, thanks!

How do I debug keras model

@surya-veer I am going through tutorial for handwritten text recognition. And to do hand written digit recognition you have constructed a Keras model as follows:

# # Creating CNN model

input_shape = (28,28,1)
number_of_classes = 10

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=input_shape))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))

model.add(Dropout(0.5))
model.add(Dense(number_of_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),metrics=['accuracy'])

model.summary()

history = model.fit(X_train, y_train,epochs=5, shuffle=True,
                    batch_size = 200,validation_data= (X_test, y_test))


model.save('digit_classifier2.h5')

I am very confused on how have you choosen these layers. I know how Conv2D works by applying filters to an image, I know what is activation function. In short I have a rough understanding of what each term means.

What I am finding it difficult is how do I know what is happening in each step of this code? For example lets take this python code:

values_List=[11,34,43]
for index, num in enumerate(values_List):
    print(index,num)

I know that line 1 initializes a list named values_List
Line 2 iterates through this list
Line 3 prints output as (index of a number , number)

This python code is easy to understand and debug. But I am confused that if there is any error inside the keras layers. How do I proceed to debug this Keras code ? How do I see output on each step inside the Keras code ?

What algorithm

Hey
did you use SVM to make this app ?