harish2704/pottan-ocr

A stupid OCR for malayalam language

Notice: This project is not relevent anymore since latest version of tesseract ocr is using same technology ( CNN-RNN models ) and it is capable of detecting complex scripts with very high accuracy . A web demo of latest tesseract ocr can be seen in the link given below

https://harish2704.github.io/ml-tesseract-demo/

pottan-ocr

A stupid OCR for malayalam language. It can be Easily configured to process any other languages with complex scripts

Client side web Demo of individual line recognition

https://harish2704.github.io/pottan-demo/

Screenshot of complete page OCR

Installation

Clone the project

git clone https://github.com/harish2704/pottan-ocr
cd pottan-ocr

Run the installer bash script to complete the installation.

• For Debian

  env DISTRO=debian ./tools/install-dependencies.sh

• For Fedora

  env DISTRO=fedora ./tools/install-dependencies.sh

• For OpenSUSE

  env DISTRO=opensuse ./tools/install-dependencies.sh

• For Ubuntu

  ./tools/install-dependencies.sh

By default, the installer will install dependencies which is necessary to run the OCR. For training the OCR, pass the string for_training as first argument to installer.

  ./tools/install-dependencies.sh for_training

Usage

1. Download latest pre-trained model file from pottan-ocr-data repository

wget 'https://github.com/harish2704/pottan-ocr-data/raw/master/crnn_11032020_171631_5.h5' -O pottan_ocr_latest.h5

2. Create configuration file

cp ./config.yaml.sample ./config.yaml

3. Run the OCR using any PNG/JPEG image

./bin/pottan ocr <trained_model.h5> <iamge_path> [ pottan_ocr_output.html ]

For more details, see the --help of bin/pottan and its subcommands

Usage:
./pottan <command> [ arguments ]

List of available commands ( See '--help' of individual command for more details ):

    extractWikiDump - Extract words from wiki xml dump ( most most of the text corpus ). Output is written to stdout.

    datagen         - Prepare training data from data/train.txt & data/validate.txt. ( Depreciated. used only for manual varification of training data )

    train           - Run the training

    ocr             - Run charector recognition with a pre-trained model and image file

Training

• Training is done using synthetic images generated on the fly using text corpus. For this to work, we should have enough fonts installed in the system. In short, fonts listed in the ./config.yaml.sample should be available in the output of command fc-list :lang=ml
  • It is also possible to write the generated images to disk. sub-command datagen does exactly this. When running training, if the images already found to exists in the cache directory( eg: point cache directory to generated images directory ), it will be used for the training instead of generating new images. This idea is used to reduce CPU load during production training sessions
• Also it is recommended to have a GPU for training the OCR.

Try training on Google colab

Open notebook

For more details, see wiki

Getting involved

• Join public Gitter chat room ( See badge on the top ) or Public Matrix chat room #pottan-ocr:matrix.org ( https://riot.im/app/#/room/#pottan-ocr:matrix.org ).
• Status, progress & pending tasks can be seen @ https://github.com/harish2704/pottan-ocr/projects/1

Credits

• Authors of http://arxiv.org/abs/1507.05717
• jieru mei who created pytorch implementation for above mentioned model. Repo https://github.com/meijieru/crnn.pytorch. The model used in Pottan-OCR is taken from this project.
• Tom and the contributes of Ocropy project ( https://github.com/tmbdev/ocropy ) which is the back-bone of pottan-ocr.
  • Code-base of pottan-ocr can do only one thing. Just convert a single line of image into single line of text.
  • Everything else including layout detection, line segmentation, output generation etc are handled by Ocropy.
  • pottan-ocr just works as core engine by replacing default engine Tesseract OCR used in the Ocropy
• Pytorch https://pytorch.org/
• Leon Chen and the Team behind KerasJS .
  • KerasJS is used to create the Web-based demo application of the OCR.
  • KerasJS does its job very well by running Keras Models in browsers with WebGL2 acceleration.
  • It also have great features such as visualizing each stages of process , which is not explored yet.

Thanks

• Swathanthra Malayalam Computing group members for evaluating and providing suggestions.
• Stackoverflow user "Yu-Yang" for answering my question

pottan-ocr's Issues

How to use this for recognize Tamil language ?

How to use this for recognize Tamil language ?

Add documentation on this.

Cleanup dataset generation using augmentor python module

How could you convert pytorch model to keras

@harish2704 I can not convert pytorch lstmcell to keras, Could you please provide some help?
Here is pytorch lstmcell,
self.att_lstm = nn.LSTMCell(1536,512)
h_att, c_att = self.att_lstm(att_lstm_input, (state[0][0], state[1][0]))
state[0][0], state[1][0] is the tensor(2, 512)

Create a public online service to quickly collect Data samples for OCR and setup automated release of trained models based on availability of new dataset

Migrate to apache singa since it has better support for devices

Apache singa seems to support both OpenCL and CUDA out of the box
https://singa.incubator.apache.org/en/index.html

port to https://github.com/dmlc/tvm for running inside wide range of devices

Create a "thnet" or "pytorch2c" application which can run on any device by using less resources

Decide what all variations should be included while rendering a text.

Some options includes

• Bold / italic version of current font
• Use many different fonts
• Change offset position ( X, Y co-ordinates ), letter spacing, letter rotation
• Custom complex jitters and noise functions

Add word-accuracy and character-error-rate

• Word Accuracy = correct-words/total-words
• Character Error Rate = Sum(edit-distance(prediction, truth))/sum(len(truth))

### Add documentation & installation instructions

* Remove dependency on system fonts.

• Fonts used for training should be part of training data
• Make it possible to render text using given ttf font file

Recognition issues

For the image given below,

output is as follows
'വികുസ്തിപ്ലീക്കൂമ്പഠ (ൃവ൭൭ന്രിക്കൂന്ന റിധ്രിത ന്ത്രജ്ജ൭ത്തിമ്രത്ഠ൬ ഉങ്കല്ലൂ'

### Setup integration with Kraken / Ocropy

This will tool will work as an engine without doing any sort of

• Layout analysis
• Column detection
• Output document generation

Etc.

It will simply do Imag with Line of text to Text conversion

Need more info on how to do the training?

I am trying to train for tamil fonts.

created this repo.
https://github.com/nithyadurai87/pottan-ocr-tamil

https://github.com/nithyadurai87/pottan-ocr-tamil/blob/master/config.yaml
added this file.

It seems the install step for training needs GPU, CUDA etc.
when installing them and enabling the NVDIA driver, my ubuntu got crashed.
reinstalled ubuntu.

Is there any way to train on cloud?
You used Floydhub.

Requesting you to add tutorial on how to train using floudhub.

Thanks.