Comparison of Machine Learning Algorithms in Public Sentiment Analysis of TAPERA Policy

Eklesia  Sihombing; Muhammad   Halmi Dar; Fitri   Aini Nasution

doi:10.46729/ijstm.v5i5.1164

Eklesia Sihombing Faculty of Science and Technology, Universitas Labuhanbatu, Sumatera Utara Indonesia
Muhammad Halmi Dar Faculty of Science and Technology, Universitas Labuhanbatu, Sumatera Utara Indonesia
Fitri Aini Nasution Faculty of Science and Technology, Universitas Labuhanbatu, Sumatera Utara Indonesia

DOI: https://doi.org/10.46729/ijstm.v5i5.1164

Keywords: Machine Learning, Naïve Bayes, Random Forest, Sentiment Analysis Support Vector Machine and Tapera.

Abstract

The rapid development of information technology has changed the way people interact and express their opinions on public policies, including the People's Housing Savings (Tapera) policy in Indonesia. People now primarily express their views openly on social media platforms like Twitter, generating a substantial amount of text data for analysis to understand public sentiment. However, the main challenge in this sentiment analysis is determining the most effective machine learning algorithm for classifying public opinion with high accuracy. This study aims to compare the performance of three machine learning algorithms, namely Naïve Bayes, Support Vector Machine, and Random Forest, in analyzing public sentiment towards the Tapera policy. This study analyzes public comment data obtained from Twitter. We measure the accuracy of each algorithm to determine its optimal performance in sentiment classification. The research method consists of several stages, starting with data collection, text preprocessing to clean and prepare data, and then applying the three algorithms to analyze sentiment. The results showed that Naïve Bayes had the highest accuracy of 69.17%, followed by Support Vector Machine with an accuracy of 68.42%, and Random Forest with an accuracy of 66.17%. This shows that Naïve Bayes is the most effective algorithm to use in sentiment analysis of public comments related to the Tapera policy, especially in the context of complex text data from social media. The conclusion of this study is that Naïve Bayes is superior in classifying public sentiment towards the Tapera policy compared to Support Vector Machine and Random Forest. As a result, this study makes a significant contribution to selecting the most appropriate machine learning algorithm for public sentiment analysis towards public policy, which in turn can help the government understand and respond to public perceptions more effectively.

Downloads

Download data is not yet available.

References

Suwardono and G. Santoso, “Peran Media Massa dan Opini Publik dalam Mendukung atau Mengancam Kesehatan Demokrasi,” J. Pendidik. Transform., vol. 02, no. 03, pp. 239–249, 2023, doi: 10.9000/jpt.v2i3.1393.

I. G. P. Megayasa, P. P. O. Mahawardana, and P. R. Nurbawa, “Analisis Sentimen berdasarkan Opini dari Media Sosial Twitter terhadap ‘Figure Pemimpin’ menggunakan Python,” J. Manaj. dan Teknol. Inf., vol. 13, no. 1, 2023, doi: 10.5281/zenodo.7934336.

A. Andirwan, V. Asmilita, M. Zhafran, A. Syaiful, and M. Beddu, “Strategi Pemasaran Digital: Inovasi untuk Maksimalkan Penjualan Produk Konsumen di Era Digital,” JIMAT J. Ilm. Multidisiplin Amsir, vol. 2, no. 1, pp. 155–166, 2023, doi: 10.62861/jimat%20amsir.v2i1.405.

U. Aulia and M. I. P. Nasution, “Memanfaatkan Data Media Sosial untuk Intelijen Kompetitif di Era Digital,” J. Informatics Busisnes, vol. 02, no. 01, pp. 78–83, 2024.

O. Manullang, C. Prianto, and N. H. Harani, “Analisis Sentimen untuk Memprediksi Hasil Calon Pemilu Presiden menggunakan Lexicon Based dan Random Forest,” J. Ilm. Inform., vol. 11, no. 02, pp. 159–169, 2023, doi: 10.33884/jif.v11i02.7987.

N. Raisa, N. Riza, and W. I. Rahayu, “Analisis Sentimen menggunakan SVM dan KNN pada Review Drama Korea di MYDRAMALIST,” JINTEKS (Jurnal Inform. Teknol. dan Sains), vol. 5, no. 4, 2023, doi: 10.51401/jinteks.v5i4.3114.

D. Pramana, M. Afdal, Mustakim, and I. Permana, “Analisis Sentimen terhadap Pemindahan Ibu Kota Negara menggunakan Algoritma Naive Bayes Classifier dan K-Nearest Neightbors,” J. Media Inform. Budidarma, vol. 7, no. 3, pp. 1306–1314, 2023, doi: 10.30865/mib.v7i3.6523.

Z. Munawar et al., Big Data Analytics: Konsep, Implementasi, dan Aplikasi Terkini, Pertama. Bandung: Kaizen Publisher, 2023.

A. Kaharudin, A. A. Supriyadi, Muhlis, H. Baitika, and M. Derryanur, “Analisis Sentimen pada Media Sosial dengan Teknik Kecerdasan Buatan Naïve Bayes: Kajian Literatur Review,” OKTAL J. Ilmu Komput. dan Sci., vol. 2, no. 6, pp. 1642–1649, 2023, [Online]. Available: https://journal.mediapublikasi.id/index.php/oktal/article/view/2944%0Ahttps://journal.mediapublikasi.id/index.php/oktal/article/download/2944/1371

B. H. Dzakiyyah, K. D. Putri, N. Y. Salsabila, T. A. Rafania, and I. F. A. Prawira, “Pemanfaatan Big Data untuk Meningkatkan Kepuasan Pelanggan Shopee,” Innov. J. Soc. Sci. Res., vol. 3, no. 5, pp. 10441–10455, 2023, doi: 10.31004/innovative.v3i5.5534.

N. Ulyah, “Analisis Strategi Pemasaran untuk Meningkatkan Penjualan Pada PT. Bhirawa Steel,” 2016. [Online]. Available: http://eprints.perbanas.ac.id/163/1/ARTIKEL ILMIAH.pdf

B. A. Putri and R. Prijadi, “Public Fund Optimization for Housing Finance (Case Study: Tabungan Perumahan Rakyat, Indonesia),” in Proceedings of the 5th International Conference on Economics, Business and Economic Education Science, ICE-BEES 2022, 2023. doi: 10.4108/eai.9-8-2022.2338624.

M. Ihsan, A. Rofiq, and Khusnudin, “Polemik Tabungan Perumahan Rakyat (Tapera): Sebuah kajian dengan pendekatan interdisipliner,” Gulawentah J. Stud. Sos., vol. 9, no. 1, pp. 72–86, 2024, doi: 10.25273/gulawentah.v9i1.20497.

C. Ariningdyah, D. Lasonda, and F. R. D. Miarsa, “Analisis Yuridis Penerapan Tabungan Perumahan Rakyat (Tapera) dalam Perspektif Asas Keadilan,” Innov. J. Soc. Sci. Res., vol. 4, no. 3, pp. 18410–18424, 2024, doi: 10.31004/innovative.v4i3.12769.

N. Haviazzahra, “Analisis Hukum Kepesertaan Pekerja Mandiri dalam Pelaksanaan Program Penyelenggaraan Tabungan Perumahan Rakyat,” Aliansi J. Hukum, Pendidik. dan Sos. Hum., vol. 1, no. 5, 2024, doi: 10.62383/aliansi.v1i5.386.

M. Pasah, M. Yohana, and H. Winata, “Urgensi Penerapan Tapera bagi Pegawai Swasta di Indonesia,” CAUSA J. Huk. dan Kewarganegaraan, vol. 5, no. 2, 2024, doi: 10.3783/causa.v2i9.2461.

B. Liu, Sentiment Analysis: Mining Opinions, Sentiments, and Emotions, 2nd Editio. Cambridge University Press, 2020.

A. Pak and P. Paroubek, “Twitter as a Corpus for Sentiment Analysis and Opinion Mining,” in Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), N. Calzolari, K. Choukri, B. Maegaard, J. Mariani, J. Odijk, S. Piperidis, M. Rosner, and D. Tapias, Eds., Valletta, Malta: European Language Resources Association (ELRA), May 2010. [Online]. Available: http://www.lrec-conf.org/proceedings/lrec2010/pdf/385_Paper.pdf

A. Suryahadi, R. Al Izzati, and D. Suryadarma, “The Impact of COVID-19 Outbreak on Poverty: An Estimation for Indonesia,” 2020. [Online]. Available: http://smeru.or.id/en/content/impact-covid-19-outbreak-poverty-estimation-indonesia

N. Rulandari, “Public Participation in Policy Making: Sentiment Analysis of TAPERA Policy on Twitter,” Ilomata Int. J. Soc. Sci., vol. 5, no. 3, 2024, doi: 10.61194/ijss.v5i3.1296.

E. D. Harahap and R. Kurniawan, “Analisis Sentimen Komentar terhadap Kebijakan Pemerintah Mengenai Tabungan Perumahan Rakyat (TAPERA) pada Aplikasi X menggunakan Metode Naïve Bayes,” J. Tek. Inform. Unika ST. Thomas, vol. 09, no. 01, 2024.

T. S. Rambe, M. N. S. Hasibuan, and M. H. Dar, “Sentiment Analysis of Beauty Product Applications using the Naïve Bayes Method,” SinkrOn, vol. 8, no. 2, pp. 980–989, 2023, doi: 10.33395/sinkron.v8i2.12303.

T. Joachims, “Text categorization with Support Vector Machines: Learning with many relevant features,” in Machine Learning: ECML-98, C. Nédellec and C. Rouveirol, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 1998, pp. 137–142.

A. M. Rahat, A. Kahir, and A. K. M. Masum, “Comparison of Naive Bayes and SVM Algorithm based on Sentiment Analysis Using Review Dataset,” in 2019 8th International Conference System Modeling and Advancement in Research Trends (SMART), 2019, pp. 266–270. doi: 10.1109/SMART46866.2019.9117512.

L. Breiman, “Random Forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.

P. Karthika, R. Murugeswari, and R. Manoranjithem, “Sentiment Analysis of Social Media Network Using Random Forest Algorithm,” in 2019 IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS), 2019, pp. 1–5. doi: 10.1109/INCOS45849.2019.8951367.

Kaggle, “Tapera Dataset,” kaggle.com. Accessed: Jun. 25, 2024. [Online]. Available: https://www.kaggle.com/datasets/unshoytable/twitter-tapera-dateset.