Sentiment Analysis of Gojek Driver Application Reviews Using Support Vector Machine and Naïve Bayes with Optuna-Based Hyperparameter Tuning
DOI:
https://doi.org/10.61255/decoding.v4i1.948Keywords:
Hyperparameter Tuning, Naïve Bayes, Optuna, Sentiment Analysis, Support Vector MachineAbstract
Purpose – This study aims to analyze user review sentiment toward the Gojek Driver application and compare the performance of two classification algorithms, Support Vector Machine (SVM) and Naïve Bayes, using Optuna as a framework for hyperparameter tuning.
Methods – The study collected and labeled user review data into positive and negative sentiment categories. Text preprocessing involved cleaning, case folding, normalization, tokenization, stopword removal, and stemming. Features were represented using TF-IDF. The dataset was then divided into training and testing sets, and SVM and Naïve Bayes models were trained using automated hyperparameter optimization with Optuna. Model performance was evaluated using accuracy, precision, recall, F1-score, and confusion matrix.
Findings – The application of SMOTE to the Optuna-tuned SVM model produced better performance than the other models tested in this study. The best model achieved an accuracy of 0.868, a highest cross-validation accuracy of 92.72%, and a weighted average F1-score of 0.87. These results indicate that SVM was more effective in handling high-dimensional TF-IDF features and complex decision boundaries.
Research implications – The findings support the use of automated sentiment analysis to assist operational decision-making and improve the quality of Gojek Driver services. The proposed approach can accelerate the identification of service-related issues and provide a basis for proactive responses to user feedback.
Originality – This study offers an original contribution by directly comparing SVM and Naïve Bayes on a Gojek Driver review dataset while applying Optuna-based hyperparameter tuning. It highlights the effect of automated tuning on both algorithms within a TF-IDF representation framework for ride-hailing service data, a topic that remains underexplored in the specific context of Gojek Driver within the local literature.
Abstract views: 13
, 
PDF downloads: 7
Downloads
References
S. Biswas and G. Poornalatha, “Opinion Mining Using Multi-Dimensional Analysis,” IEEE Access, vol. 11, pp. 25906–25916, 2023, doi: 10.1109/ACCESS.2023.3256521.
T. Sugihartono and R. R. C. Putra, “PENERAPAN METODE SUPPORT VECTOR MACHINE DALAM CLASSIFIKASI ULASAN PENGGUNA APLIKASI MOBILE JKN,” SKANIKA Sist. Komput. Dan Tek. Inform., vol. 7, no. 2, pp. 144–153, Jul. 2024, doi: 10.36080/skanika.v7i2.3193.
A. Suryana, A. I. Purnamasari, and I. Ali, “MENGOPTIMALKAN KEPUASAN PENGGUNA: ANALISIS SENTIMEN REVIEW APLIKASI GRAB DI INDONESIA,” JATI J. Mhs. Tek. Inform., vol. 8, no. 3, pp. 3396–3404, May 2024, doi: 10.36040/jati.v8i3.9688.
N. Amat-Lefort, F. Barravecchia, and L. Mastrogiacomo, “Quality 4.0: big data analytics to explore service quality attributes and their relation to user sentiment in Airbnb reviews,” Int. J. Qual. Reliab. Manag., vol. 40, no. 4, pp. 990–1008, Mar. 2023, doi: 10.1108/IJQRM-01-2022-0024.
S. V, G. V. Varshini, N. M, D. R, V. K. Kaliappan, and J. Li, “Customer Feedback Automation and Classification using Sentiment Analysis, Robotic Process Automation and Uipath,” in Recent Trends in Data Science and its Applications, Kattankulathur, Tamil Nadu, India: River Publishers, 2023, pp. 899–907. doi: 10.13052/rp-9788770040723.175.
K. Huda, S. D. Pohan, and Y. Herlina, “PENERAPAN PEMBOBOTAN TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY DAN ALGORITMA K-NEAREST NEIGHBOR UNTUK ANALISIS ULASAN HOTEL DI SITUS TRIPADVISOR,” J. Inform. Dan Tek. Elektro Terap., vol. 12, no. 3, Aug. 2024, doi: 10.23960/jitet.v12i3.4800.
S. Siswanto, Z. Mar’ah, A. S. D. Sabir, T. Hidayat, F. A. Adhel, and W. S. Amni, “The Sentiment Analysis Using Naïve Bayes with Lexicon-Based Feature on TikTok Application,” J. Varian, vol. 6, no. 1, pp. 89–96, Nov. 2022, doi: 10.30812/varian.v6i1.2205.
A. Safira and F. N. Hasan, “ANALISIS SENTIMEN MASYARAKAT TERHADAP PAYLATER MENGGUNAKAN METODE NAIVE BAYES CLASSIFIER,” ZONAsi J. Sist. Inf., vol. 5, no. 1, pp. 59–70, Jan. 2023, doi: 10.31849/zn.v5i1.12856.
A. Jaha, N. Satya Deepthi, G. Suryakanth, and G. S. K. Eswari, “Text Sentiment Analysis Using Naïve Baye’s Classifier,” Int. J. Comput. Trends Technol., vol. 68, no. 4, pp. 261–265, Apr. 2020, doi: 10.14445/22312803/IJCTT-V68I4P141.
K. Hanif, A. Fadllullah, N. R. Muntiari, and I. A. Fahrezi, “A Comparative Sentiment Analysis of Computer Engineering Student Feedback Using Decision Trees and SVM,” J. Inotera, vol. 10, no. 1, pp. 71–82, Feb. 2025, doi: 10.31572/inotera.Vol10.Iss1.2025.ID436.
F. Fersellia, E. Utami, and A. Yaqin, “Sentiment Analysis of Shopee Food Application User Satisfaction Using the C4.5 Decision Tree Method,” Sinkron, vol. 8, no. 3, pp. 1554–1563, Jul. 2023, doi: 10.33395/sinkron.v8i3.12531.
Y. Liu and H. Kou, “Student career guidance using sentiment analysis and decision tree models,” Int. J. Inf. Commun. Technol., vol. 26, no. 38, pp. 1–21, 2025, doi: 10.1504/IJICT.2025.149290.
A. A. Sulaeman, M. Danny, S. Butsianto, and S. Pratama, “Sentiment Analysis on Social Media X (Twitter) Against ChatGBT Using the K-Nearest Neighbors Algorithm,” Brill. Res. Artif. Intell., vol. 4, no. 1, pp. 265–275, Jul. 2024, doi: 10.47709/brilliance.v4i1.4105.
D. Wijati, P. D. Atika, S. Setiawati, and R. Rasim, “Sentiment Analysis of Application Reviews using the K-Nearest Neighbors (KNN) Algorithm,” PIKSEL Penelit. Ilmu Komput. Sist. Embed. Log., vol. 12, no. 1, pp. 209–218, Mar. 2024, doi: 10.33558/piksel.v12i1.9490.
K. Nugroho, E. Winarno, D. R. I. M. Setiadi, and O. Farooq, “Enhanced multi-lingual Twitter sentiment analysis using hyperparameter tuning k-nearest neighbors,” Bull. Electr. Eng. Inform., vol. 13, no. 6, pp. 4327–4334, Dec. 2024, doi: 10.11591/eei.v13i6.7265.
A. Pambudi and S. Suprapto, “Effect of Sentence Length in Sentiment Analysis Using Support Vector Machine and Convolutional Neural Network Method,” IJCCS Indones. J. Comput. Cybern. Syst., vol. 15, no. 1, p. 21, Jan. 2021, doi: 10.22146/ijccs.61627.
Muthia Sakhdiah, Admi Salma, Dony Permana, and Dina Fitria, “Sentiment Analysis Using Support Vector Machine (SVM) of ChatGPT Application Users in Play Store,” UNP J. Stat. Data Sci., vol. 2, no. 2, pp. 151–158, May 2024, doi: 10.24036/ujsds/vol2-iss2/158.
T. A. Khan, R. Sadiq, Z. Shahid, M. M. Alam, and M. Mohd Su’ud, “Sentiment Analysis using Support Vector Machine and Random Forest,” J. Inform. Web Eng., vol. 3, no. 1, pp. 67–75, Feb. 2024, doi: 10.33093/jiwe.2024.3.1.5.
F. M. Sarimole and K. Kudrat, “Analisis Sentimen Terhadap Aplikasi Satu Sehat Pada Twitter Menggunakan Algoritma Naive Bayes Dan Support Vector Machine: Analisis Sentimen,” J. Sains Dan Teknol., vol. 5, no. 3, pp. 783–790, Feb. 2024, doi: 10.55338/saintek.v5i3.2702.
R. Setiawan and F. Dewi, “Analysis of Twitter User Sentiment on Presidential Candidate Anies Baswedan Using Naïve Bayes Algorithm,” Ilk. J. Ilm., vol. 15, no. 3, pp. 473–487, Dec. 2023, doi: 10.33096/ilkom.v15i3.1775.473-487.
W. Saidi, A. E. Abderrahmani, and K. Satori, “EFFECTIVE COMPARATIVE EVALUATION OF SENTIMENT ANALYSIS USING PAIRED T-TEST: A PERFORMANCE STUDY OF SUPERVISED METHODS,” J. Southwest Jiaotong Univ., vol. 58, no. 5, 2023, doi: 10.35741/issn.0258-2724.58.5.28.
A. Thumpati and Y. Zhang, “Towards Optimizing Performance of Machine Learning Algorithms on Unbalanced Dataset,” in Artificial Intelligence & Applications, Academy & Industry Research Collaboration Center, Oct. 2023, pp. 169–183. doi: 10.5121/csit.2023.131914.
S. N. Almuayqil, M. Humayun, N. Z. Jhanjhi, M. F. Almufareh, and N. A. Khan, “Enhancing Sentiment Analysis via Random Majority Under-Sampling with Reduced Time Complexity for Classifying Tweet Reviews,” Electronics, vol. 11, no. 21, p. 3624, Nov. 2022, doi: 10.3390/electronics11213624.
E. Hokijuliandy, H. Napitupulu, and Firdaniza, “Application of SVM and Chi-Square Feature Selection for Sentiment Analysis of Indonesia’s National Health Insurance Mobile Application,” Mathematics, vol. 11, no. 17, p. 3765, Sep. 2023, doi: 10.3390/math11173765.
R. Rahmanda and E. B. Setiawan, “Word2Vec on Sentiment Analysis with Synthetic Minority Oversampling Technique and Boosting Algorithm,” J. RESTI Rekayasa Sist. Dan Teknol. Inf., vol. 6, no. 4, pp. 599–605, Aug. 2022, doi: 10.29207/resti.v6i4.4186.
D. A. Kristiyanti, S. A. Sanjaya, V. C. Tjokro, and J. Suhali, “Dealing imbalance dataset problem in sentiment analysis of recession in Indonesia,” IAES Int. J. Artif. Intell. IJ-AI, vol. 13, no. 2, p. 2060, Jun. 2024, doi: 10.11591/ijai.v13.i2.pp2060-2072.
R. Obiedat et al., “Sentiment Analysis of Customers’ Reviews Using a Hybrid Evolutionary SVM-Based Approach in an Imbalanced Data Distribution,” IEEE Access, vol. 10, pp. 22260–22273, 2022, doi: 10.1109/ACCESS.2022.3149482.
G. Ryan, P. Katarina, and D. Suhartono, “MBTI Personality Prediction Using Machine Learning and SMOTE for Balancing Data Based on Statement Sentences,” Information, vol. 14, no. 4, p. 217, Apr. 2023, doi: 10.3390/info14040217.
