kushagratomar/attention-based-image-caption-generation

Attention-Based-Image-Caption-Generation

The image captioning is utilized to develop the explanations of the sentences describing the series of scenes captured in the image or picture forms. The practice of using image captioning is vast although it is a tedious task for the machine to learn what a human is capable of. The model must be built in a way such that when it reads the scene, it recognizes and reproduce to-the-point captions or descriptions. The generated descriptions must be semantically and syntactically accurate. Hence, availability of Artificial Intelligence (AI) and Machine Learning algorithms viz. Natural Language Processing (NLP), Deep Learning (DL) etc. makes the task easier. In the proposed paper, anew introduction to attention mechanism called Bahdanau’s along with Encoder-Decoder architecture is being used so as to reflect the captions of the image. A pre-trained Convolutional Neural Network (CNN) called InceptionV3 architecture is used to gather the features of images and then a Recurrent Neural Network (RNN) called Gated Recurrent Unit (GRU) architecture so as to develop captions is utilized. The results obtained from this model is trained on Flickr8k dataset with the improvement in accuracy around 10% with the present state of the art.