ai4bharat/indicnlp-transliteration

Codebase for Indic-Transliteration using Seq2Seq RNN. For latest repo with Transformer-based models, check: https://github.com/AI4Bharat/IndicXlit

IndianNLP-Transliteration

Project Website | Demo UI | Python Library

The main goal of this project is to create open source input tools for content creation in under-represented languages in India.
It started in collaboration with Story Weaver a non-profit working towards foundational literary education for children, supported by Google's AI for Social Good initiative.

Most languages in India do not have digital presence due to an underdeveloped ecosystem. One of the major bottlenecks in content creation and language adoption, is difficulty to input text in several native Indian languages. Lack of stable input tools in underserved languages is huge barrier for creating digital content and NLP datasets in these languages.

Supported Languages

• Bengali - বাংলা
• Gujarati - ગુજરાતી
• Hindi - हिंदी
• Kannada - ಕನ್ನಡ
• Konkani Goan - कोंकणी
• Maithili - मैथिली
• Malayalam - മലയാളം
• Marathi - मराठी
• Panjabi Eastern - ਪੰਜਾਬੀ
• Sindhi - سنڌي‎
• Sinhala - සිංහල
• Telugu - తెలుగు
• Tamil - தமிழ்
• Urdu - اُردُو

---

Repository Usage

For Attributions and Contributions lists, check here 🖖

Training Procedures

This repository is developed to facilate easier experimentation with different network architecture models, reformulated objectives with minimal effort and highly tinkerable, rather than a offshelf library.

A Condensed standalone version of a simple model training, inferencing and accuracy computation is created as jupyter notebook.

Pythonic Library

Pythonic transliteration library is available as Python Package Index and also under github releases.
Follow usages in apps readme.

---

NeuralNet Models

Transliteration models for languages are made available as releases, in a easy deployable way.

All the NN models (along with metadata) of Xlit - Transliteration are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

---

Datasets

Datasets created as part of the project for languages Maithili, Konkani, Hindi are made available as JSON files under downloads.

Xlit - Transliteration Datasets by Story Weaver & AI4Bharat are licensed under a Creative Commons Attribution 4.0 International License.

Kindly attribute if you use the dataset for your research or products

---

Contact

If you have benefited by our datasets/models/services or got motivated by our works, we would like to hear from you.

email: [email protected]

---

indicnlp-transliteration's Issues

Indic-to-Latin Transliteration

Hello! Thanks for your great efforts!

Is there a way to transliterate from an Indian language to Latin characters?

Not an issue but a suggestion transliteration from orignal script to Roman in english

Nice library for Indian sub continent languages for various purposes. I have a suggestion if we pass a original language script it should transliterate in roman english which I have not found in the library which will help in lot of cases. It is doing from roman english to selected language.

Didn't understand how to use the UI.

Might seem stupid. But does it translate stuff written in english to Indian languages or is it that you write a word in say Hindi in English font and it provides you the Indian option of it. Please do help me out. TIA!

Support for Batch Processing and Inference on GPU

Hi, thanks for the transliteration api. It's helpful.

1. Is it possible to transliterate words in batches rather than one at a time?
2. Also, can there be a flag in API where the model can use GPU if available?

Support transliteration for Odia language

Support transliteration for Odia language. I am interested if anybody guides me on how to implement it.