kevkibe/african-whisper

🚀 Framework for seamless fine-tuning of Whisper model on a multi-lingual dataset and deployment to prod.

african-whisper's Issues

Translation and Transcription in Fast API endpoint

Currently the endpoint deployed when running src/deployment/main.py only supports the default function: transcription.

Enhancement: Add tag for installing just deployment dependencies.

Description:

Adding a tag for installing only the deployment dependencies of the "AfricanWhisper" package.

Problem:

The current installation process for the "AfricanWhisper" package installs all modules, including those required for training and deployment.
A lightweight installation containing only the deployment dependencies to reduce package size and avoid unnecessary dependencies is optimal.

Solution:

To address this issue, introducing a tag, such as [deployment], that users can append to the package name when installing via pip.
For example:

• pip install africanwhisper installs the entire package, including training and deployment modules.
• pip install africanwhisper[deployment] installs only the components necessary for deployment, excluding training-related dependencies.

Feature: PEFT FInetuning

Allow users to either fully fine-tune their models or use PEFT to fine-tune a portion of the parameters

Bug: Error installing package on Kaggle.

Bug

• Installing africanwhisper[training], africanwhisper[deployment] or africanwhisper[all] on Kaggle raises the error below:

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.10/site-packages/aiohttp-3.9.1.dist-info/METADATA'

Solution

• Execute this command before installing the package

!rm /opt/conda/lib/python3.10/site-packages/aiohttp-3.9.1.dist-info -rdf

Update: Inference Pipeline using Gradio/Streamlit

Add inference for the trained model using gradio/streamlit for the UI.

Enhancement: Use Terraform for deployment in Google VM, AWS EC2 Instance

Description:

Terraform files into our deployment process to facilitate the provisioning and management of infrastructure resources on Google Cloud Platform (GCP) VMs and Amazon Web Services (AWS) EC2 Instances.

Problem:

Deployment process for provisioning infrastructure resources on GCP VMs and AWS EC2 Instances may involve manual steps or scripts, which can be error-prone, time-consuming, and difficult to maintain.

Proposed Solution:

• Integrating Terraform into our deployment workflow, to automate the provisioning and configuration of infrastructure resources on GCP VMs and AWS EC2 Instances

`deployment.speech_inference` module not working.

The example using deployment.speech_inference module is not working, on the python package distribution.
You will get this error working on a notebook.

ModuleNotFoundError: No module named 'deployment.faster_whisper'

Just a build problem and it will be fixed a couple of hours from now.
You can use the deployment.speech_inference module on the CLI regardless, follow the steps here

Feature: Stream processing for datasets.

Currently only batch streaming is supported which is memory consuming.
The feature is to enable stream processing to save memory for large datasets.

Feature: Docker containerization of the trained model

Add a feature to package the trained model as a docker image for deployment to a cloud platform

Implement Optional Parameter Specification for Seq2SeqTrainingArguments

Enhance user experience by allowing flexible specification of training arguments within Seq2SeqTrainingArguments, enabling users to customize training processes according to their needs.

Feature: Deploying inference endpoint for fine-tuned models

A feature to allow user's to deploy REST API endpoint to get transcriptions using the fine-tuned model.

Inference from PEFT fine-tuned models

Inference pipeline to support both fully fine-tuned and peft fine-tuned

Option for translation and transcription on audio files in Gradio demo UI

Bug: Investigate model saving twice in Notebook run.

• Training run with parameter processing_task= "automatic-speech-recognition" saves the model twice with one directory named /content/automatic-speech-recognition and another /content/transcribe with the same content.
• Investigate the prepare_model method and the trainer.train.

Feature: Support for Timestamp in Inference API endpoint

Enable the endpoint deployed at endpoint /transcribe using src/deployment/main.py to support timestamps and return timestamp as well as text as output.

Bug: Error with Translating/Transcribing Audio from Microphone on Gradio UI.