From Machine Learning to Transformers: A Comparative Study on Emotion Classification of Tweets
DOI:
https://doi.org/10.21015/vtcs.v14i1.2236Abstract
In this era of social media, it is essential that policymakers, businessmen, and researchers focus on understanding the sentiments of the people through their expressions on social media. An analytical comparison of several well-known machine learning (ML) algorithms along with Transformer-based models has been exploited in this study to determine the emotions and sentiments of people through their tweets. ML algorithms included in the study consider classical models—Random Forest, SVM, ID3, Naive Bayes, KNN, MLP—and transformer models, namely BERT-Base and BERT-Large. These algorithms have been applied to a carefully curated dataset of tweets that have been labeled for their emotion, such as joy, fear, anger, and sadness. Evaluation metrics, including precision, accuracy, kappa score, F1 score, and recall, have been used to compare and analyze the performance of each algorithm. The findings of this paper reveal that both Random Forest and SVM showed the best accuracy, at 79.4% and 79.1% while Random Forest slightly outperformed SVM in accuracy. In Transformers, BERT-Base outperforms not only BERT-Large but also the performance of all ML algorithms with an accuracy of 86.3%. The proposed study offers important insights for future applications and studies in the area of emotion classification. It also highlights the importance of choosing a suitable algorithm for tasks related to text categorization.
References
O. Adwan, M. Al-Tawil, A. Huneiti, R. Shahin, A. A. Zayed, and R. Al-Dibsi, “Twitter sentiment analysis approaches: A survey,” Int. J. Emerg. Technol. Learn. (iJET), vol. 15, no. 15, pp. 79–93, 2020.
A. D. Dubey, “Twitter sentiment analysis during COVID-19 outbreak,” SSRN Electron. J., 2020.
H. Huang, A. A. Zavareh, and M. B. Mustafa, “Sentiment analysis in e-commerce platforms: A review of current techniques and future directions,” IEEE Access, vol. 11, pp. 90367–90382, 2023.
L. Mandloi and R. Patel, “Twitter sentiments analysis using machine learning methods,” in Proc. Int. Conf. Emerging Technol. (INCET), 2020, pp. 1–5.
K. H. Manguri, R. N. Ramadhan, and P. R. M. Amin, “Twitter sentiment analysis on worldwide COVID-19 outbreaks,” Kurdistan J. Appl. Res., pp. 54–65, 2020.
A. Go, R. Bhayani, and L. Huang, “Twitter sentiment classification using distant supervision,” CS224N Project Report, Stanford Univ., 2009.
A. Pak and P. Paroubek, “Twitter as a corpus for sentiment analysis and opinion mining,” in Proc. LREC, 2010, pp. 1320–1326.
X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” in Adv. Neural Inf. Process. Syst., vol. 28, 2015.
J. Devlin, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv:1810.04805, 2018.
Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019.
H. Wang, J. Li, H. Wu, E. Hovy, and Y. Sun, “Pre-trained language models and their applications,” Engineering, vol. 25, pp. 51–65, 2023.
A. A. Chowdhury et al., “Sentiment analysis of COVID-19 vaccination from survey responses in Bangladesh,” 2021.
Y. Zhang and B. Wallace, “A sensitivity analysis of convolutional neural networks for sentence classification,” arXiv:1510.03820, 2015.
E. Kouloumpis, T. Wilson, and J. Moore, “Twitter sentiment analysis: The good the bad and the OMG!,” in Proc. Int. AAAI Conf. Web Soc. Media, vol. 5, no. 1, pp. 538–541, 2011.
L. Breiman, “Random forests,” Mach. Learn., vol. 45, pp. 5–32, 2001.
V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, pp. 273–297, 1995.
J. R. Quinlan, “Induction of decision trees,” Mach. Learn., vol. 1, pp. 81–106, 1986.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, 1986.
H. Zhang, “The optimality of naive Bayes,” AA, vol. 1, no. 2, p. 3, 2004.
D. W. Aha, D. Kibler, and M. K. Albert, “Instance-based learning algorithms,” Mach. Learn., vol. 6, pp. 37–66, 1991.
M. P. LaValley, “Logistic regression,” Circulation, vol. 117, no. 18, pp. 2395–2399, 2008.
J. D. M.-W. C. Kenton and L. K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019.
A. Mitra and S. Mohanty, “Sentiment analysis using machine learning approaches,” in Emerging Technol. Data Mining Inf. Security (IEMIS), vol. 2, pp. 63–68, 2020.
Y. Wang, J. Guo, C. Yuan, and B. Li, “Sentiment analysis of Twitter data,” Appl. Sci., vol. 12, no. 22, p. 11775, 2022.
N. Braig et al., “Machine learning techniques for sentiment analysis of COVID-19-related Twitter data,” IEEE Access, vol. 11, pp. 14778–14803, 2023.
D. Antonakaki, P. Fragopoulou, and S. Ioannidis, “A survey of Twitter research: Data model, graph structure, sentiment analysis and attacks,” Expert Syst. Appl., vol. 164, p. 114006, 2021.
F. Barbieri, L. E. Anke, and J. Camacho-Collados, “XLM-T: Multilingual language models in Twitter for sentiment analysis and beyond,” arXiv:2104.12250, 2021.
Z. B. Nezhad and M. A. Deihimi, “Twitter sentiment analysis from Iran about COVID-19 vaccine,” Diabetes Metab. Syndr., vol. 16, no. 1, p. 102367, 2022.
A. S. Neogi et al., “Sentiment analysis and classification of Indian farmers’ protest using Twitter data,” Int. J. Inf. Manage. Data Insights, vol. 1, no. 2, p. 100019, 2021.
Y. Qi and Z. Shabrina, “Sentiment analysis using Twitter data: Comparative application of lexicon- and machine-learning-based approaches,” Soc. Netw. Anal. Mining, vol. 13, no. 1, p. 31, 2023.
X. Liu et al., “Emotion classification for short texts: An improved multi-label method,” Humanit. Soc. Sci. Commun., vol. 10, no. 1, pp. 1–9, 2023.
I. Ameer et al., “Multi-label emotion classification in texts using transfer learning,” Expert Syst. Appl., vol. 213, p. 118534, 2023.
Z. Li et al., “Word-level emotion distribution for short text emotion classification,” Knowl.-Based Syst., vol. 227, p. 107163, 2021.
C. Singla et al., “An optimized deep learning model for emotion classification in tweets,” Comput. Mater. Continua, vol. 70, no. 3, 2022.
N. Jamal et al., “A deep learning-based approach for emotions classification in imbalanced tweets,” ACM Trans. Asian Low-Resour. Lang. Inf. Process., vol. 20, no. 3, pp. 1–16, 2021.
A. Chiorrini et al., “Emotion and sentiment analysis of tweets using BERT,” in EDBT/ICDT Workshops, 2021, pp. 1–7.
A. Glenn, P. LaCasse, and B. Cox, “Emotion classification of Indonesian tweets using bidirectional LSTM,” Neural Comput. Appl., vol. 35, no. 13, pp. 9567–9578, 2023.
J. F. Raisa et al., “A review on Twitter sentiment analysis approaches,” in Proc. Int. Conf. ICT Sustainable Develop. (ICICT4SD), 2021, pp. 375–379.
F. Rustam et al., “Performance comparison of supervised machine learning models for COVID-19 tweets sentiment analysis,” PLoS One, vol. 16, no. 2, p. e0245909, 2021.
A. Poornima and K. S. Priya, “Comparative sentiment analysis of sentence embedding using machine learning techniques,” in Proc. Int. Conf. Adv. Comput. Commun. Syst. (ICACCS), 2020, pp. 493–496.
J. Xue et al., “Twitter discussions and emotions about the COVID-19 pandemic: Machine learning approach,” J. Med. Internet Res., vol. 22, no. 11, p. e20550, 2020.
M. H. Algifari and E. D. Nugroho, “Emotion classification of Indonesian tweets using BERT embedding,” J. Appl. Inform. Comput., vol. 7, no. 2, pp. 172–176, 2023.
L. He, “Enhanced Twitter sentiment analysis with dual joint classifier integrating RoBERTa and BERT architectures,” Front. Phys., vol. 12, p. 1477714, 2024.
A. Sathish et al., “Intelligent emotion sensing using BERT BiLSTM and generative AI,” Sci. Rep., vol. 15, no. 1, pp. 1–22, 2025.
T. Nijhawan, G. Attigeri, and T. Ananthakrishna, “Stress detection using natural language processing and machine learning over social interactions,” J. Big Data, vol. 9, no. 1, p. 33, 2022.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC-By) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This work is licensed under a Creative Commons Attribution License CC BY
