Performance Evaluation of Machine Learning Algorithms in Aspect-Based Sentiment Analysis on E-Commerce User Reviews
Abstract
The rapid growth of the e-commerce industry in Indonesia has resulted in a significant surge in the number of user reviews available on various digital platforms. These reviews contain valuable information about customer experiences related to price, product quality, service, delivery, and applications. However, the massive volume of data and its unstructured nature pose challenges in extracting relevant information. Aspect-Based Sentiment Analysis (ABSA) presents an approach that can provide deeper insights by identifying sentiment towards specific aspects within a review, rather than just the overall general sentiment. This study aims to evaluate the performance of several machine learning algorithms, namely Naïve Bayes, Support Vector Machine (SVM), Random Forest, and K-Nearest Neighbors (KNN), in implementing ABSA on e-commerce user reviews in Indonesia. The dataset used consists of 20,000 user reviews of the Shopee and Tokopedia applications obtained through a crawling process on the Google Play Store. The data is processed through several stages: text preprocessing, aspect and sentiment annotation, model training, and performance evaluation using accuracy, precision, recall, and F1-Score metrics. The evaluation results showed differences in performance among the tested algorithms. Naïve Bayes achieved an accuracy of 82.5%, KNN achieved 84.6%, Random Forest 87.1%, while SVM provided the best performance with an accuracy of 89.3% and an F1-Score of 88.3%. This difference in performance indicates that algorithms that are better able to handle high-dimensional text representations, such as SVM, are superior in aspect-based sentiment classification compared to other methods. Thus, this study not only provides a comprehensive overview of the effectiveness of machine learning algorithms in sentiment analysis in the e-commerce sector but also provides a practical basis for developing recommendation systems, improving customer service, and enhancing user experience strategies on digital platforms. This research is expected to serve as a reference in the application of machine learning to support the growth of the e-commerce industry in Indonesia.
Downloads
References
I. T. Administration and U. S. D. of Commerce, “Indonesia - eCommerce,” trade.gov. Accessed: Apr. 05, 2025. [Online]. Available: https://www.trade.gov/country-commercial-guides/indonesia-ecommerce
A. Z. Yonatan, “Nilai Transaksi E-Commerce Indonesia Capai Rp487 Triliun pada 2024,” GoodStats. Accessed: Apr. 04, 2025. [Online]. Available: https://goodstats.id/article/nilai-transaksi-e-commerce-indonesia-capai-rp487-triliun-pada-2024-Vqv7l
P. M. Alamdari, N. J. Navimipour, M. Hosseinzadeh, A. A. Safaei, and A. Darwesh, “A Systematic Study on the Recommender Systems in the E-Commerce,” IEEE Access, vol. 8, pp. 115694–115716, 2020, doi: 10.1109/access.2020.3002803.
A. Bernovskis, D. Sceulovs, and A. Stibe, “Society 5.0: Shaping the future of e-commerce,” Journal of Open Innovation: Technology, Market, and Complexity, vol. 10, no. 4, p. 100391, 2024, doi: 10.1016/j.joitmc.2024.100391.
Y. Liu, J. Lu, J. Yang, and F. Mao, “Sentiment analysis for e-commerce product reviews by deep learning model of Bert-BiGRU-Softmax,” Mathematical Biosciences and Engineering, vol. 17, no. 6, pp. 7819–7837, 2020, doi: 10.3934/mbe.2020398.
M. A. Kassem, A. A. Abohany, A. A. A. El-Mageed, and K. M. Hosny, “A novel deep learning model for detection of inconsistency in e-commerce websites,” Neural Comput Appl, vol. 36, no. 17, pp. 10339–10353, 2024, doi: 10.1007/s00521-024-09590-5.
S. Safar, B. R. Jose, and T. Santhanakrishnan, “An Improved Recommendation System with Aspect-Based Sentiment Analysis,” in Responsible Data Science, J. Mathew, G. Santhosh Kumar, D. P., and J. M. Jose, Eds., Singapore: Springer Nature Singapore, 2022, pp. 75–87.
T. S. Rambe, M. N. S. Hasibuan, and M. H. Dar, “Sentiment Analysis of Beauty Product Applications using the Naïve Bayes Method,” SinkrOn, vol. 8, no. 2, pp. 980–989, 2023, doi: 10.33395/sinkron.v8i2.12303.
R. Thomas and J. R. Jeba, “A novel framework for an intelligent deep learning based product recommendation system using sentiment analysis (SA),” Automatika, vol. 65, no. 2, pp. 410–424, 2024, doi: 10.1080/00051144.2023.2295148.
P. Sasikala and L. Mary Immaculate Sheela, “Sentiment analysis of online product reviews using DLMNN and future prediction of online product using IANFIS,” J Big Data, vol. 7, no. 1, 2020, doi: 10.1186/s40537-020-00308-7.
N. Chintalapudi, G. Battineni, M. Di Canio, G. G. Sagaro, and F. Amenta, “Text mining with sentiment analysis on seafarers’ medical documents,” International Journal of Information Management Data Insights, vol. 1, no. 1, p. 100005, 2021, doi: 10.1016/j.jjimei.2020.100005.
A.-M. Iddrisu, S. Mensah, F. Boafo, G. R. Yeluripati, and P. Kudjo, “A sentiment analysis framework to classify instances of sarcastic sentiments within the aviation sector,” International Journal of Information Management Data Insights, vol. 3, no. 2, p. 100180, 2023, doi: 10.1016/j.jjimei.2023.100180.
L. Yang, Y. Li, J. Wang, and R. S. Sherratt, “Sentiment Analysis for E-Commerce Product Reviews in Chinese Based on Sentiment Lexicon and Deep Learning,” IEEE Access, vol. 8, pp. 23522–23530, 2020, doi: 10.1109/access.2020.2969854.
E. S. P. Krishna et al., “Enhancing E-commerce recommendations with sentiment analysis using MLA-EDTCNet and collaborative filtering,” Sci Rep, vol. 15, no. 1, p. 6739, Feb. 2025, doi: 10.1038/s41598-025-91275-7.
P. Savci and B. Das, “Prediction of the customers’ interests using sentiment analysis in e-commerce data for comparison of Arabic, English, and Turkish languages,” Journal of King Saud University - Computer and Information Sciences, vol. 35, no. 3, pp. 227–237, 2023, doi: 10.1016/j.jksuci.2023.02.017.
N. Pughazendi, P. V Rajaraman, and M. H. Mohammed, “Graph Sample and Aggregate Attention Network optimized with Barnacles Mating Algorithm based Sentiment Analysis for Online Product Recommendation,” Appl Soft Comput, vol. 145, p. 110532, 2023, doi: 10.1016/j.asoc.2023.110532.
N. R. Bhowmik, M. Arifuzzaman, M. R. H. Mondal, and M. S. Islam, “Bangla Text Sentiment Analysis Using Supervised Machine Learning with Extended Lexicon Dictionary,” Natural Language Processing Research, vol. 1, no. 3–4, p. 34, 2021, doi: 10.2991/nlpr.d.210316.001.
J.-J. Wu and S.-T. Chang, “Exploring customer sentiment regarding online retail services: A topic-based approach,” Journal of Retailing and Consumer Services, vol. 55, p. 102145, 2020, doi: 10.1016/j.jretconser.2020.102145.
M. P. Geetha and D. Karthika Renuka, “Improving the performance of aspect based sentiment analysis using fine-tuned Bert Base Uncased model,” International Journal of Intelligent Networks, vol. 2, pp. 64–69, 2021, doi: 10.1016/j.ijin.2021.06.005.
E. Sihombing, M. Halmi Dar, and F. Aini Nasution, “Comparison of Machine Learning Algorithms in Public Sentiment Analysis of TAPERA Policy,” International Journal of Science, Technology & Management, vol. 5, no. 5, pp. 1089–1098, 2024, doi: 10.46729/ijstm.v5i5.1164.
L. Zhu, M. Xu, Y. Bao, Y. Xu, and X. Kong, “Deep learning for aspect-based sentiment analysis: a review,” PeerJ Comput Sci, vol. 8, pp. e1044–e1044, Jul. 2022, doi: 10.7717/peerj-cs.1044.
H. H. Do, P. W. C. Prasad, A. Maag, and A. Alsadoon, “Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review,” Expert Syst Appl, vol. 118, pp. 272–299, 2019, doi: 10.1016/j.eswa.2018.10.003.
M. H. D. T. Ari Bangsa, S. Priyanta, and Y. Suyanto, “Aspect-Based Sentiment Analysis of Online Marketplace Reviews Using Convolutional Neural Network,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 14, no. 2, p. 123, 2020, doi: 10.22146/ijccs.51646.
A. Karimi, L. Rossi, and A. Prati, “Adversarial Training for Aspect-Based Sentiment Analysis with BERT,” in 2020 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 8797–8803. doi: 10.1109/ICPR48806.2021.9412167.
J. K. Prathi, P. K. Raparthi, and M. V. Gopalachari, “Real-Time Aspect-Based Sentiment Analysis on Consumer Reviews BT - Data Engineering and Communication Technology,” K. S. Raju, R. Senkerik, S. P. Lanka, and V. Rajagopal, Eds., Singapore: Springer Nature Singapore, 2020, pp. 801–810.
A. Daza, N. D. González Rueda, M. S. Aguilar Sánchez, W. F. Robles Espíritu, and M. E. Chauca Quiñones, “Sentiment Analysis on E-Commerce Product Reviews Using Machine Learning and Deep Learning Algorithms: A Bibliometric Analysis, Systematic Literature Review, Challenges and Future Works,” International Journal of Information Management Data Insights, vol. 4, no. 2, p. 100267, 2024, doi: 10.1016/j.jjimei.2024.100267.
L. Davoodi, J. Mezei, and M. Heikkilä, “Aspect-based sentiment classification of user reviews to understand customer satisfaction of e-commerce platforms,” Electronic Commerce Research, 2025, doi: 10.1007/s10660-025-09948-4.
O. Bellar, A. Baina, and M. Ballafkih, “Sentiment Analysis: Predicting Product Reviews for E-Commerce Recommendations Using Deep Learning and Transformers,” Mathematics, vol. 12, no. 15, p. 2403, 2024, doi: 10.3390/math12152403.
H. J. Alantari, I. S. Currim, Y. Deng, and S. Singh, “An empirical comparison of machine learning methods for text-based sentiment analysis of online consumer reviews,” International Journal of Research in Marketing, vol. 39, no. 1, pp. 1–19, 2022, doi: 10.1016/j.ijresmar.2021.10.011.
P. Rasappan, M. Premkumar, G. Sinha, and K. Chandrasekaran, “Transforming sentiment analysis for e-commerce product reviews: Hybrid deep learning model with an innovative term weighting and feature selection,” Information Processing & Management, vol. 61, no. 3, p. 103654, 2024, doi: 10.1016/j.ipm.2024.103654.
H. Fang, G. Jiang, and D. Li, “Sentiment analysis based on Chinese BERT and fused deep neural networks for sentence-level Chinese e- commerce product reviews,” SYSTEMS SCIENCE & CONTROL ENGINEERING, 2022, doi: 10.1080/21642583.2022.2123060.
R. Liu, H. Wang, and Y. Li, “AgriMFLN: Mixing Features LSTM Networks for Sentiment Analysis of Agricultural Product Reviews,” Applied Sciences, vol. 13, no. 10, p. 6262, 2023, doi: 10.3390/app13106262.
Copyright (c) 2025 International Journal of Science, Technology & Management
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
