sakshi025/byteverse_notabot

notabot

video-pitch: https://drive.google.com/file/d/1ZtAmCCxFcUiur0q-Sx1Yha52ohMPgUJ6/view?usp=share_link;

Image to Audio Description for visually impaired

The main idea behind this project is to create an image captioning system that can generate a textual description of an image, which can then be converted into an audio description to assist blind individuals in understanding the content of the image. This can be accomplished using deep learning techniques and pre-trained models.

The process can be broken down into three main steps:

Image Captioning: A pre-trained deep learning model is used to generate a textual description of the image. This is done by feeding the image through a convolutional neural network (CNN) to extract visual features, and then passing those features through a recurrent neural network (RNN) to generate the textual description. The model is trained on a large dataset of images and corresponding captions, so it can learn to generate accurate and descriptive captions.We trained VGG-16 model as RNN

Text-to-Speech: For the Text-to-Speech part of the project, a software library called gTTS (Google Text-to-Speech) was used to convert the textual description generated by the image captioning model into an audio description. gTTS uses Google's artificial intelligence to generate natural-sounding speech from written text. This process involves training the AI model on a large dataset of speech samples to ensure the resulting audio output is of high quality and realistic.

Integration: The image captioning and TTS systems are then integrated into a single application, which can take input from a webcam and output an audio description of the image in real-time. This application can be useful for blind individuals who want to understand the content of images, such as photos or videos, that they would not otherwise be able to access.

Dataset used: Flickr 8K

Dependencies requied

pip install flask tensorflow keras Pillow nltk gTTS