mmesomachukwu/cohort8-ransome-kuti-ladipo

A Machine Learning System for Sign Language Video Translation and Speech Generation into Low-Resource Languages (LRLs) in Nigeria. AISaturdayLagos (Cohort- 8 Ransom-Kuti-Ladipo)

NSL-2-AUDIO

NSL-2-AUDIO

NSL-2-AUDIO is an open-source Automatic Sign Language Translation system, specifically designed to translate Nigerian Sign Language (NSL) into one of the Low-Resource Languages (LRLs) spoken in Nigeria."
Explore the docs »

View Demo · Report Bug · Request Feature

Important

This project is currently in a very early development/experimental stage. There are a lot of unimplemented/broken features at the moment. Contributions are welcome to help out with the progress!

Table of Contents

• About
• Demos
• System Architecture
• Quick Start
• Getting Started
• Roadmap
• Contributing
• License
• Contact
• Acknowledgments

About

Overview:
This project is dedicated to the development of an Automatic Sign Language Translation system, with a specific focus on translating Nigerian Sign Language (NSL) into one of the Low-Resource Languages (LRLs) spoken in Nigeria. The primary objective is to address the communication challenges faced by the NSL community and contribute to inclusivity and employment opportunities in education and the workforce.

Significance:
Effective communication is a cornerstone of societal cohesion, and this project addresses a critical gap in the integration of the NSL community into the broader society. Sign language, while vital for the hearing-impaired, often faces limitations in bridging the gap with the larger community. This project seeks to overcome these limitations and promote a more inclusive and understanding society.

Potential Applications:

• Education:
  • Integration into schools and educational institutions to support hearing-impaired students.
  • Facilitation of communication in educational settings, creating a more inclusive learning environment.
• Workforce:
  • Facilitating communication in the workplace, creating job opportunities for NSL speakers.
  • Promotion of diversity and inclusion in professional environments.
• Community Involvement:
  • Empowering NSL speakers to engage more actively in community activities.
  • Bridging the gap between the NSL community and the broader society.

You can read the project proposal here, Project Proposal

Built With

• 
• 
• 
• 
• 

(back to top)

Note

Currently, our support is exclusively for the Yoruba language.
However, we aspire to expand our services to include other low-resource Nigerian languages such as Igbo and Hausa in the future.

Demos

This is a Temporary demo

System Architecture

Architecture of Inference Pipeline

System Design

(back to top)

Quick Start

The easiest way to run the project locally:

# Clone this repository
$ git clone https://github.com/rileydrizzy/NSL_2_AUDIO

# Go into the repository
$ cd NSL_2_AUDIO

# Go to app directory
$ cd app

Getting Started

The project is structured into three distinct parts, each housed in separate directories as outlined in the project proposal. The initial phase involves translating sign language into English text, followed by the second phase, which focuses on translating the English text into Yoruba text. The final segment entails taking the translated Yoruba text and converting it into generated Yoruba audio.

The signa2text directory is dedicated to the process of translating sign language into English text. Meanwhile, the linguify_yb directory serves the purpose of transforming English text into Yoruba text. Finally, the yb2audio directory is designated for utilizing the translated audio to generate Yoruba audio.

In the app directory, hold the logic for the application. This directory serves as the core repository for all the essential functionalities and trained models required for the smooth operation of the application. Here, you'll find the code responsible for handling user interactions, processing of video data, managing state, and performing inference.

To access any of the directories, navigate into the respective directory and adhere to the specified prerequisites in the respective directory README.

(back to top)

💡 Roadmap

Contributing

We welcome contributions to enhance NSL-2-AUDIO capabilities and improve its performance. To contribute, please see the CONTRIBUTING.md file for steps.

License

NSL-2-AUDIO is released under the MIT License. See the LICENSE file for more information.

Contact

If you have questions or need assistance, feel free to reach out to:

Name: Ipadeola Ezekiel Ladipo
Email: [email protected]
GitHub: @rileydrizzy
Linkdeln: Ipadeola Ladipo

Acknowledgments

I would like to acknowledge the outstanding contributions of :

Name: Afonja Tejumade (Mentor)
Email: [email protected]
GitHub: @tejuafonja

(back to top)

---