ashrafsau/bard-bangla-article-classifier

BARD Bangla Article Classifier

Video Demo: bit.ly/BARD_VIDEO_DEMO
Dataset Link: bit.ly/BARD_DATASET

Dataset Details

Category	No. of Documents	No. of Words	Average Sentences per Document	Average words per Sentence
State	242860	57019465	18.50	13.356
Economy	18982	4915141	20.18	13.378
International	32203	7096111	18.47	12.493
Entertainment	31293	6706563	21.70	10.236
Sports	50888	12397415	22.80	11.069

Abstract

In the literature article classification is well studied, where several supervised leaning models have been proposed by utilizing large textual data corpus. Despite several comprehensive textual dataset available for different language, only a few small datasets available for Bangla articles. As a result, a couple works address the Bangla document classification and could not able to learn sophisticated supervised learning model. In this work, we curated a large dataset of Bangla article which help us to train several supervised learning model using some sophisticate features, such as word2vec. As word2vec preserves semantic features, it shows superior performance in text classification. Moreover, Neural Network with word2vec features shows promising performance compared to other state-of-the-art-work in the text classification.
Furthermore, we deployed our proposed Bangla content classifier as a web application, which is accessible here. and the video demo of this application is available here. Additionally, we open-sourced the BARD dataset and source code of this work in this repository.

Statistical Analysis

Figure 01: Most frequent words in each categories.

We performed textual statistical analysis on the BARDdataset articles and results are presented in Fig. 1 and 2. Thefrequency distributions of the top 20 most frequent words foreach of the five categories are depicted in Fig. 1. From thisanalysis, we can easily identify that all the categories havethe similar most frequent words. In other words, these frequentwords do not help to categorize the articles. Hence, we removearound 25 most frequent from all the articles and performedthe statistical analysis again on the filtered dataset, which ispresented in Fig. 2. Now, this filtered frequency distributiondepicted that each category has some unique distribution ofwords, which may contribute to categorize the articles.

Figure 02: Most frequent words in each categories after removing stop words.