Sentiment Analysis of User Reviews for AI Applications: Evaluating SVM, Logistic Regression, and Random Forest

Authors

  • Rivana Dwi Cahyani
  • Putri Taqwa Prasetyaningrum

DOI:

https://doi.org/10.63158/journalisi.v8i1.1366

Keywords:

sentiment analysis, Random Forest, Support Vector Machine, AI applications, machine learning

Abstract

The rapid growth of AI applications such as CICI, GROK, and Gemini has resulted in a large volume of user reviews on platforms like the Google Play Store, making sentiment analysis a critical tool for understanding user perceptions. This study compares the performance of three machine learning models: Random Forest, Support Vector Machine (SVM), and Logistic Regression in classifying sentiments in 3,500 Indonesian-language reviews. A hybrid feature extraction approach, combining sentiment lexicons with TF-IDF, was applied to improve sentiment classification accuracy. The models were evaluated based on accuracy, precision, recall, and F1-score. Results indicated that all models achieved an accuracy greater than 96%, with Random Forest providing the most consistent and accurate results, achieving an overall accuracy of 99.62%. While SVM excelled in classifying positive and negative sentiments, it faced challenges with neutral reviews due to the ambiguity and overlap in sentiment expression. Logistic Regression also showed strong performance, especially on structured reviews. The findings suggest that Random Forest is the most robust and reliable model for sentiment analysis, particularly in handling diverse AI application reviews. These results offer practical insights for developers seeking to improve application performance by leveraging sentiment analysis on user feedback.

Downloads

Download data is not yet available.

References

[1] S. A. Putra and A. Wijaya, “Sentiment Analysis of Artificial Intelligence (AI) on Twitter Social Media Using Lexicon-Based Method (Analisis Sentimen Artificial Intelligence (Ai) Pada Media Sosial Twitter Menggunakan Metode Lexicon Based),” JuSiTik: J. Sist. Teknol. Inform. Komunik., vol. 7, no. 1, pp. 21–28, 2023.

[2] B. H. Nugroho, “Comparison of AI Capabilities of Grok, ChatGPT, and Gemini in Social Media Content Analysis (Perbandingan Kemampuan AI Grok, ChatGPT, dan Gemini dalam Analisis Konten Media Sosial),” LogicLink, vol. 2, no. 1, pp. 56–69, 2025.

[3] A. G. Budianto, A. Trisno, E. Suryo, and G. Rudi, “Comparison of the Performance of Support Vector Machine (SVM) and Logistic Regression Algorithms for Sentiment Analysis of Retail Application Users on Android (Perbandingan Performa Algoritma Support Vector Machine (SVM) dan Logistic Regression untuk Analisis Sentimen Pengguna Aplikasi Retail di Android),” J. Sains Dan Informatika, vol. 10, no. November, pp. 1–10, 2024.

[4] I. T. Julianto and L. Lindawati, “Sentiment Analysis of the Academic Information System at the Garut Institute of Technology (Analisis Sentimen terhadap Sistem Informasi Akademik Institut Teknologi Garut),” J. Algoritma, vol. 19, no. 1, pp. 458–468, 2022.

[5] S. F. Kadir and A. Fairuzabadi, “Sentiment Analysis of Shopee Reviews on Google Play Using TF-IDF and Logistic Regression (Analisis Sentimen Ulasan Shopee di Google Play dengan TF-IDF dan Logistic Regression),” RIGGS: J. Artif. Intell. Digital Bus., vol. 4, no. 2, pp. 7940–57945, 2025.

[6] B. Kholifah, I. Thoib, N. Sururi, and N. D. Kurnia, “Sentiment Analysis of Public Opinion on Online Transportation Service Issues Using Lexicon-Based InSet with Logistic Regression (Analisis Sentimen Warganet terhadap Isu Layanan Transportasi Online Berbasis InSet Lexicon menggunakan Logistic Regression),” KLIK-KUMPULAN JURNAL ILMU KOMPUTER, vol. 11, no. 1, pp. 14–25, 2024.

[7] S. Butsianto and A. M. Rifa'i, “Sentiment Analysis of Jamsostek Application Reviews Using SVM, Random Forest, and Logistic Regression (Analisis Sentimen Ulasan Aplikasi Jamsostek dengan SVM, Random Forest, dan Logistic Regression),” J. Informatika Ekonomi Bisnis, pp. 700–706, 2025, doi: 10.37034/infeb.v7i3.1266.

[8] S. N. Adhan, G. N. A. Wibawa, D. C. Arisona, I. Yahya, and R. Ruslan, “Sentiment Analysis of Wattpad Application Reviews on Google Play Store Using Random Forest (Analisis Sentimen Ulasan Aplikasi Wattpad Di Google Play Store Dengan Metode Random Forest),” AnoaTIK: J. Teknol. Inform. Komp., vol. 2, no. 1, pp. 6–15, 2024.

[9] P. A. Effendi and T. Ernawati, “Sentiment Analysis of Hay Day Game Application Reviews Using Random Forest Algorithm (Analisis Sentimen Ulasan Aplikasi Game Hay Day Menggunakan Algoritma Random Forest),” J. Informatika Dan Teknik Elektro Terapan, vol. 13, no. 3S1, 2025.

[10] M. D. Hendriyanto, A. A. Ridha, and U. Enri, “Sentiment Analysis of Mola Application Reviews on Google Play Store Using Support Vector Machine Algorithm (Analisis Sentimen Ulasan Aplikasi Mola Pada Google Play Store Menggunakan Algoritma Support Vector Machine),” J. Inf. Technol. Comput. Sci., vol. 5, no. 1, pp. 1–7, 2022.

[11] A. A. Munandar, F. Farikhin, and C. E. Widodo, “Sentiment Analysis of Online Learning Applications Using SVM Classification (Sentimen Analisis Aplikasi Belajar Online Menggunakan Klasifikasi SVM),” JOINTECS (J. Inf. Technol. Comput. Sci.), vol. 8, no. 2, p. 77, 2023, doi: 10.31328/jointecs.v8i2.4747.

[12] M. B. Prayogi and G. Masitoh, “Sentiment Analysis of Alfagift Application User Reviews Using Random Forest (Analisis Sentimen Ulasan Pengguna Aplikasi Alfagift Menggunakan Random Forest),” JISKA (J. Informatika Sunan Kalijaga), vol. 10, no. 2, pp. 158–170, 2025.

[13] A. F. Anjani, D. Anggraeni, and I. M. Tirta, “Implementation of Random Forest Using SMOTE for Sentiment Analysis of Sister for Students UNEJ Application Reviews (Implementasi Random Forest Menggunakan SMOTE untuk Analisis Sentimen Ulasan Aplikasi Sister for Students UNEJ),” Jurnal Nasional Teknologi Dan Sistem Informasi, vol. 9, no. 2, pp. 163–172, 2023.

[14] N. O. Adiwijaya, M. F. Al Abror, T. Dharmawan, and M. A. Hidayat, “Optimizing the Thesis Topic Recommendation Model Based on Student Academic Performance Using SMOTE (Optimasi Model Rekomendasi Topik Skripsi berdasarkan Performa Akademik Mahasiswa menggunakan SMOTE),” Proc. Seminar Nasional Teknik Elektro, Sistem Informasi, dan Teknik Informatika (SNESTIK), vol. 5, no. 1, pp. 83–90, Jul. 2025.

[15] R. Wahyudi et al., “Sentiment Analysis on Grab Application Reviews on Google Play Store Using Support Vector Machine (Analisis sentimen pada review aplikasi grab di google play store menggunakan support vector machine),” Jurnal Informatika, vol. 8, no. 2, pp. 200–207, 2021.

[16] U. Kulsum, M. Jajuli, and N. Sulistiyowati, “Sentiment Analysis of WeTV Application on Google Play Store Using Support Vector Machine Algorithm (Analisis Sentimen Aplikasi WETV di Google Play Store Menggunakan Algoritma Support Vector Machine),” J. Appl. Informatics Comput., vol. 6, no. 2, pp. 205–212, 2022.

[17] E. Eskiyaturrofikoh and R. R. Suryono, “Sentiment Analysis of Application X on Google Play Store Using Naive Bayes and Support Vector Machine (SVM) Algorithms (Analisis sentimen aplikasi x pada google play store menggunakan algoritma naïve bayes dan support vector machine (svm)),” JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 9, no. 3, pp. 1408–1419, 2024.

[18] R. I. Alhaqq, I. Made, K. Putra, Y. Ruldeviyani, I. M. K. Putra, and Y. Ruldeviyani, “Sentiment Analysis of MySAPK BKN Application Usage on Google Play Store (Analisis Sentimen terhadap Penggunaan Aplikasi MySAPK BKN di Google Play Store),” Jurnal Nasional Teknik Elektro Dan Teknologi Informasi, vol. 11, no. 2, 2022.

[19] T. Tinaliah and T. Elizabeth, “Sentiment Analysis of PrimaKu Application Reviews Using Support Vector Machine Method (Analisis Sentimen Ulasan Aplikasi PrimaKu Menggunakan Metode Support Vector Machine),” JATISI (J. Teknik Informatika Dan Sistem Informasi), vol. 9, no. 4, pp. 3436–3442, 2022.

[20] M. Fauzi et al., “Implementation of Machine Learning for Weather Prediction Using Support Vector Machine (Implementasi Machine Learning Untuk Memprediksi Cuaca Menggunakan Support Vector Machine),” J. Ilm. Komputasi, vol. 23, no. 1, pp. 45–50, 2024, doi: 10.32409/jikstik.23.1.3499.

[21] K. A. Rokhman, B. Berlilana, and P. Arsi, “Comparison of Support Vector Machine and Decision Tree Methods for Sentiment Analysis of Reviews on Online Transportation Applications (Perbandingan metode support vector machine dan decision tree untuk analisis sentimen review komentar pada aplikasi transportasi online),” J. Inf. Syst. Manag. (JOISM), vol. 2, no. 2, pp. 1–7, 2021.

[22] F. A. Larasati, D. E. Ratnawati, and B. T. Hanggara, “Sentiment Analysis of Dana Application Reviews Using Random Forest Method (Analisis Sentimen Ulasan Aplikasi Dana dengan Metode Random Forest),” J. Pengembang. Teknol. Inform. Ilmu Komput., vol. 6, no. 9, pp. 4305–4313, 2022.

[23] O. I. Gifari, M. Adha, I. R. Hendrawan, and F. F. S. Durrand, “Sentiment Analysis of Film Reviews Using TF-IDF and Support Vector Machine (Analisis Sentimen Review Film Menggunakan TF-IDF dan Support Vector Machine),” J. Inf. Technol., vol. 2, no. 1, pp. 36–40, 2022.

[24] P. A. Nugroho, N. Sucahyo, and I. Kurniati, “Sentiment Analysis on Twitter Social Media to Assess Public Response to the Prakerja Card Selection (Sentimen Analisis pada Sosial Media Twitter untuk Menilai Respon Masyarakat terhadap Seleksi Kartu Prakerja),” J. Teknol. Inform. dan Komput. MH. Thamrin, vol. 9, no. 1, pp. 72–83, 2023.

[25] S. A. Putra and A. Wijaya, “Sentiment Analysis of Artificial Intelligence (AI) on Twitter Social Media Using Lexicon-Based Method (Analisis Sentimen Artificial Intelligence (Ai) Pada Media Sosial Twitter Menggunakan Metode Lexicon Based),” JuSiTik: J. Sist. Teknol. Inform. Komunik., vol. 7, no. 1, pp. 21–28, 2023.

Cover Volume 8 Issue 1, 2026

Downloads

Published

2026-02-10

Issue

Vol. 8 No. 1 (2026): February

Section

Articles

License

Copyright (c) 2026 Journal of Information Systems and Informatics

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors Declaration

  1. The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
  2. The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
  3. The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
  4. The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
  5. The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
  6. The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
  7. If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
  8. The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights

How to Cite

[1]
R. D. Cahyani and P. T. Prasetyaningrum, “Sentiment Analysis of User Reviews for AI Applications: Evaluating SVM, Logistic Regression, and Random Forest”, journalisi, vol. 8, no. 1, pp. 1–27, Feb. 2026, doi: 10.63158/journalisi.v8i1.1366.

Most read articles by the same author(s)