Mohsinul Kabir, Obayed Bin Mahfuz, Syed Rifat Raiyan, Hasan Mahmud, Md Kamrul Hasan
The analysis of consumer sentiment, as expressed through reviews, can provide a wealth of insight regarding the quality of a product. While the study of sentiment analysis has been widely explored in many popular languages, relatively less attention has been given to the Bangla language, mostly due to a lack of relevant data and cross-domain adaptability. To address this limitation, we present BanglaBook, a large-scale dataset of Bangla book reviews consisting of 158,065 samples classified into three broad categories: positive, negative, and neutral. We provide a detailed statistical analysis of the dataset and employ a range of machine learning models to establish baselines including SVM, LSTM, and Bangla-BERT. Our findings demonstrate a substantial performance advantage of pre-trained models over models that rely on manually crafted features, emphasizing the necessity for additional training resources in this domain. Additionally, we conduct an in-depth error analysis by examining sentiment unigrams, which may provide insight into common classification errors in under-resourced languages like Bangla. Our codes and data are publicly available at https://github.com/mohsinulkabir14/BanglaBook.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.9331 | Bangla-BERT (large) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.9106 | Random Forest (word 2-gram + word 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.9064 | Bangla-BERT (base-uncased) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.9053 | SVM (word 2-gram + word 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.9043 | Random Forest (word 1-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8978 | Logistic Regression (char 2-gram + char 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8964 | Logistic Regression (word 2-gram + word 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8723 | XGBoost (char 2-gram + char 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8663 | Multinomial NB (word 2-gram + word 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8651 | XGBoost (word 2-gram + word 3-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8564 | Multinomial NB (BoW) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.8519 | SVM (word 1-gram) |
| Sentiment Analysis | BanglaBook | Weighted Average F1-score | 0.0991 | LSTM (GloVe) |