Comparative Analysis of IndoBERT and BiLSTM For Public Sentiment Classification Toward The Indonesian National Police on Youtube
DOI:
https://doi.org/10.61255/decoding.v4i2.1144Keywords:
BiLSTM, IndoBERT, Indonesian national police, Sentiment classification, YouTube commentsAbstract
Purpose – This study aimed to compare the performance of IndoBERT and Bidirectional Long Short-Term Memory (BiLSTM) in classifying public sentiment toward the Indonesian National Police (INP) based on YouTube comments. This study sought to identify a robust sentiment classification model to support text-based public perception monitoring, particularly under a highly imbalanced sentiment distribution.
Method – YouTube comments were collected using the YouTube Data API. A total of 8,268 raw comments were obtained, and 7,197 comments were retained as the final dataset after preprocessing, automatic pseudo-labeling, and confidence filtering using a 0.5 threshold. To address concerns regarding threshold selection, an additional sensitivity analysis was conducted using confidence thresholds of 0.65 and 0.75. The experiment applied a dual-track preprocessing pipeline, cost-sensitive learning through class-weighted loss, bootstrap confidence interval analysis, and BiLSTM preprocessing ablation.
Findings – The results show that IndoBERT achieved stronger performance than BiLSTM. IndoBERT obtained an accuracy of 92.92% and a Macro-F1 Score of 0.8548, whereas BiLSTM achieved an accuracy of 76.11% and a Macro-F1 Score of 0.6124. Bootstrap analysis showed a Macro-F1 difference of 0.2424, with a 95% confidence interval of 0.1870 to 0.2959, indicating that IndoBERT’s advantage was statistically significant. Sensitivity analysis also confirmed that IndoBERT consistently outperformed BiLSTM across all the tested thresholds.
Research Implications – The findings indicate that IndoBERT is more suitable for Indonesian sentiment classification in public perception monitoring than other models. However, because the dataset labels were generated using a BERT-based classifier, the evaluation may contain architectural circularity that favors the IndoBERT model. Future studies should use human-annotated gold-standard data and broader cross-platform validations.
Originality – This study provides a comparative evaluation of transformer-based and recurrent models using sensitivity analysis, bootstrap testing, cost-sensitive learning, and pre-processing ablation under imbalanced sentiment conditions.
Abstract views: 25
, 
PDF downloads: 4
Downloads
References
Republic of Indonesia, “Law of the Republic of Indonesia Number 2 of 2002 concerning the Indonesian National Police,” 2002.
T. R. Tyler, “Enhancing police legitimacy,” The ANNALS of the American Academy of Political and Social Science, vol. 593, no. 1, pp. 84–99, 2004, doi: 10.1177/0002716203262627.
D. A. Fauzi, P. R. Nurajijah, and Engkus, “Analisis sentimen masyarakat terhadap aplikasi Digital Korlantas Polri pada ulasan Google Play Store menggunakan model IndoBERT,” Jurnal Pendidikan Sosial dan Humaniora, vol. 5, no. 1, pp. 1181–1204, 2025.
S. Riyadi, L. K. Salsabila, C. Damarjati, and R. A. Karim, “Sentiment analysis of YouTube users on Blackpink Kpop group using IndoBERT,” INTENSIF: Jurnal Ilmiah Penelitian dan Penerapan Teknologi Sistem Informasi, vol. 8, no. 2, pp. 233-245, 2024.
A. Rustamaji, R. R. Huizen, and D. P. Hostiadi, “Sentiment analysis for hotel reviews using Snowball and VADER,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), vol. 14, no. 2, 2025.
N. L. Kirana, L. Muflikhah, and Indriati, “Analisis sentimen ulasan aplikasi SP4N LAPOR! dengan IndoBERT dan koreksi ejaan berbasis Levenshtein Distance,” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 10, no. 4, 2026.
R. Tangke, D. T. Salaki, W. W. Kalengkongan, and E. Ketaren, “Analisis sentimen aplikasi TikTok menggunakan algoritma Support Vector Machine (SVM) dan Random Forest,” Jurnal TIMES, vol. 13, no. 2, pp. 53-62, 2024.
M. Khadapi and V. M. Pakpahan, “Analisis sentimen berbasis jaringan LSTM dan BERT terhadap diskusi Twitter tentang Pemilu 2024,” JUKI: Jurnal Komputer dan Informatika, vol. 6, no. 2, pp. 130–137, 2024.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM and other neural network architectures,” Neural Networks, vol. 18, no. 5–6, pp. 602–610, 2005, doi: 10.1016/j.neunet.2005.06.042.
I. K. Wijaya and R. Artana, “Analisis sentimen berbahasa Inggris dengan metode LSTM studi kasus berita online pariwisata Bali,” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 11, no. 6, pp. 1325–1334, 2024.
W. Astuti, B. Irawan, and N. A. Ramdhan, “Analisis sentimen terhadap isu pemblokiran thrifting pada platform TikTok menggunakan Bidirectional Long Short-Term Memory,” ELKOM: Jurnal Elektronika dan Komputer, vol. 18, no. 2, pp. 332–339, 2025.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, pp. 4171–4186.
B. Wilie, K. Vincentio, G. I. Winata, S. Cahyawijaya, X. Li, Z. Y. Lim, S. Soleman, R. Mahendra, P. Fung, S. Bahar, and A. Purwarianti, “IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding,” in Proc. of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, 2020, pp. 843–857.
P. Sayarizki, Hasmawati, and H. Nurrahmi, “Implementation of IndoBERT for sentiment analysis of Indonesian presidential candidates,” IndoJC, vol. 9, no. 2, pp. 61–72, 2024.
I. G. N. L. Wijayakusuma, “Perbandingan kinerja IndoBERT dan mBERT untuk deteksi berita hoaks politik dalam bahasa Indonesia,” Jurnal Sains dan Teknologi (Undiksha), vol. 14, no. 1, pp. 114–123, 2025.
M. Fahrezi, Y. B. Pratama, and A. Pramudiyantoro, “Analisis sentimen debat publik Pilpres 2024 menggunakan metode algoritma LSTM dan IndoBERT pada platform YouTube,” JPIM: Jurnal Penelitian Ilmiah Multidisipliner, vol. 2, no. 3, 2025.
F. Salsabilla and A. Witanti, “Analisis sentimen akhir masa jabatan Presiden Jokowi pada media sosial X menggunakan Naïve Bayes,” SKANIKA, vol. 8, no. 1, pp. 106-115, 2025.
M. Z. Sarwani, M. Khoiron, and M. Udin, “Optimization of the Naïve Bayes classifier algorithm using cost-sensitive learning to detect lung diseases with an imbalanced dataset,” Journal of Artificial Intelligence and Software Engineering, vol. 5, no. 1, pp. 332–338, 2025.
E. M. O. N. Haryanto, A. K. A. Estetikha, and R. A. Setiawan, “Implementasi SMOTE untuk mengatasi imbalanced data pada sentimen analisis hotel di Nusa Tenggara Barat,” Informasi Interaktif, vol. 7, no. 1, 2022.
L. N. Hayati, F. Y. Randana, and H. Darwis, “An in-depth exploration of sentiment analysis on Hasanuddin Airport using machine learning approaches,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 9, no. 2, pp. 195-208, 2025.
C. Ramadhan, V. Atina, and H. Permatasari, “Analisis perbandingan model CNN dan IndoBERT dalam sentimen berita politik Indonesia,” in Prosiding Seminar Nasional Teknologi Informasi dan Bisnis (SENATIB), pp. 110-118, 2025.
Google Developers, “YouTube Data API Overview,” Google for Developers. Available: https://developers.google.com/youtube/v3/getting-started.
Google Developers, “YouTube API Services Terms of Service,” Google for Developers. Available: https://developers.google.com/youtube/terms/api-services-terms-of-service.
M. Z. Rahman, Y. A. Sari, and N. Yudistira, “Analisis sentimen Tweet COVID-19 menggunakan Word Embedding dan metode Long Short-Term Memory (LSTM),” Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer, vol. 5, no. 11, pp. 5120–5127, 2021.
H. Rabbani, “PySastrawi: Indonesian stemmer. Python port of PHP Sastrawi project,” GitHub repository. Available: https://github.com/har07/PySastrawi.
M. Hugol, “indonesia-bert-sentiment-classification,” Hugging Face model repository. Available: https://huggingface.co/mdhugol/indonesia-bert-sentiment-classification.
F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
T. Wolf et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 38–45.
D. R. Alghifari, M. Edi, and L. Firmansyah, “Implementasi Bidirectional LSTM untuk analisis sentimen terhadap layanan Grab Indonesia,” Jurnal Manajemen Informatika (JAMIKA), vol. 12, no. 2, pp. 89–99, 2022.
P. A. Riyantoko, T. M. Fahrudin, D. A. Prasetya, T. Trimono, and T. D. Timur, “Analisis sentimen sederhana menggunakan algoritma LSTM dan BERT untuk klasifikasi data spam dan non-spam,” Prosiding Seminar Nasional Sains Data, vol. 2, no. 1, pp. 103–111, 2022.
D. Khairani, A. P. N. S. Ginting, dan R. B. Syafi'i, "Pengaruh Tahapan Preprocessing Terhadap Model IndoBERT dan IndoBERTweet Untuk Mendeteksi Emosi Pada Komentar Akun Berita Instagram," Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 11, no. 4, pp. 887-894, 2024.
D. Maynard and M. Greenwood, “Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis,” in Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), 2014.
Y. Yunitasari, A. Musdholifah, and A. K. Sari, “Sarcasm detection for sentiment analysis in Indonesian tweets,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 13, no. 1, pp. 53–62, 2019.
