UTRNet: High-Resolution Urdu Text Recognition

Official Implementation of the paper "UTRNet: High-Resolution Urdu Text Recognition In Printed Documents"

The Poster:

Using This Repository

Environment

Python 3.7
Pytorch 1.9.1+cu111
Torchvision 0.10.1+cu111
CUDA 11.4

Installation

Clone the repository

git clone https://github.com/abdur75648/UTRNet-High-Resolution-Urdu-Text-Recognition.git

Install the requirements

conda create -n urdu_ocr python=3.7
conda activate urdu_ocr
pip3 install -r requirements.txt -f https://download.pytorch.org/whl/torch_stable.html

Running the code

Training

python3 train.py --train_data path/to/LMDB/data/folder/train/ --valid_data path/to/LMDB/data/folder/val/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --exp_name UTRNet-Large --num_epochs 100 --batch_size 8

Testing

CUDA_VISIBLE_DEVICES=0 python test.py --eval_data path/to/LMDB/data/folder/test/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth

Character-wise Accuracy Testing

To create character-wise accuracy table in a CSV file, run the following command

CUDA_VISIBLE_DEVICES=0 python3 char_test.py --eval_data path/to/LMDB/data/folder/test/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC  --saved_model saved_models/UTRNet-Large/best_norm_ED.pth

Visualize the result by running char_test_vis

Reading individual images

To read a single image, run the following command

CUDA_VISIBLE_DEVICES=0 python3 read.py --image_path path/to/image.png --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC  --saved_model saved_models/UTRNet-Large/best_norm_ED.pth

Visualisation of Salency Maps

To visualize the salency maps for an input image, run the following command

python3 vis_salency.py --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth --vis_dir vis_feature_maps --image_path path/to/image.pngE

Dataset

Create your own lmdb dataset

pip3 install fire
python create_lmdb_dataset.py --inputPath data/ --gtFile data/gt.txt --outputPath result/train

The structure of data folder as below.

data
├── gt.txt
└── test
    ├── word_1.png
    ├── word_2.png
    ├── word_3.png
    └── ...

At this time, gt.txt should be {imagepath}\t{label}\n
For example

test/word_1.png label1
test/word_2.png label2
test/word_3.png label3
...

Downloads

Trained Models

Datasets

UTRSet-Real
UTRSet-Synth
IIITH (Updated) (Original)
UPTI (Source)
UrduDoc - Will be made available subject to the execution of a no-cost license agreement. Please contact the authors for the same.

Text Detection (Supplementary)

The text detection inference code & model based on ContourNet is here. As mentioned in the paper, it may be integrated with UTRNet for a combined text detection+recognition and hence an end-to-end Urdu OCR.

Synthetic Data Generation using Urdu-Synth (Supplementary)

The UTRSet-Synth dataset was generated using a custom-designed robust synthetic data generation module - Urdu Synth.

End-To-End Urdu OCR Webtool

This tool was developed by integrating the UTRNet (https://abdur75648.github.io/UTRNe) with a text detection model (YoloV8 finetuned on UrduDoc) for end-to-end Urdu OCR.

The application is deployed on Hugging Face Spaces and is available for a live demo. You can access it here. If you prefer to run it locally, you can clone its repository and follow the instructions given there - Repo.

Note: This version of the application uses a YoloV8 model for text detection. The original version of UTRNet uses ContourNet for this purpose. However, due to deployment issues, we have opted for YoloV8 in this demo. While YoloV8 is as accurate as ContourNet, it offers the advantages of faster processing and greater efficiency.

Updates

01/01/21 - Project Initiated
21/11/22 - Abstract accepted at WIDAFIL-ICFHR 2022
12/12/22 - Repository Created
20/12/22 - Results Updated
19/04/23 - Paper accepted at ICDAR 2023
23/08/23 - Poster presentation at ICDAR 2023
31/08/23 - Webtool made available
31/01/24 - Updated Webtool (with YoloV8) made available via HuggingFace here

Acknowledgements

This repository is based on deep-text-recognition-benchmark
We acknowledge the Rekhta Foundation and the personal collections of Arjumand Ara for the scanned images and Noor Fatima and Mohammed Usman for the dataset and the manual transcription.
We also thank all the members of the Computer Vision Group, IIT Delhi for their support and guidance.

Contact

Note

This is an official repository of the project. The copyright of the dataset, code & models belongs to the authors. They are for research purposes only and must not be used for any other purpose without the author's explicit permission.

Citation

If you use the code/dataset, please cite the following paper:

@InProceedings{10.1007/978-3-031-41734-4_19,
		author="Rahman, Abdur
		and Ghosh, Arjun
		and Arora, Chetan",
		editor="Fink, Gernot A.
		and Jain, Rajiv
		and Kise, Koichi
		and Zanibbi, Richard",
		title="UTRNet: High-Resolution Urdu Text Recognition in Printed Documents",
		booktitle="Document Analysis and Recognition - ICDAR 2023",
		year="2023",
		publisher="Springer Nature Switzerland",
		address="Cham",
		pages="305--324",
		isbn="978-3-031-41734-4",
		doi="https://doi.org/10.1007/978-3-031-41734-4_19"
}

License

. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License for Noncommercial (academic & research) purposes only and must not be used for any other purpose without the author's explicit permission.

abdur75648 / utrnet-high-resolution-urdu-text-recognition Goto Github PK