ANALISIS SENTIMEN ALFAGIFT DENGAN MODEL NAIVE BAYES DAN PENYEIMBANGAN DATA SMOTE

Farhan Kurniansyah; Rudi Kurniawan; Bani Nurhakim; Ade Rizki Rinaldi

Authors

Farhan Kurniansyah STMIK IKMI Cirebon, Indonesia
Rudi Kurniawan STMIK IKMI Cirebon, Indonesia
Bani Nurhakim STMIK IKMI Cirebon, Indonesia
Ade Rizki Rinaldi STMIK IKMI Cirebon, Indonesia

Keywords:

Analisis Sentimen, Naïve Bayes, SMOTE,, TF-IDF, Alfagift, NLP

Abstract

Perkembangan layanan digital mendorong peningkatan pemanfaatan analisis sentimen untuk memahami opini pengguna terhadap aplikasi mobile. Alfagift, sebagai aplikasi e-commerce ritel modern, menerima ribuan ulasan pengguna yang mencerminkan pengalaman positif maupun keluhan operasional. Namun, ulasan tersebut umumnya memiliki distribusi sentimen yang tidak seimbang, di mana ulasan positif lebih dominan dibandingkan ulasan negatif. Kondisi ini menyebabkan model klasifikasi cenderung bias, sehingga performa dalam mendeteksi sentimen minoritas menjadi rendah. Penelitian ini bertujuan untuk menganalisis pengaruh penerapan teknik Synthetic Minority Oversampling Technique (SMOTE) terhadap peningkatan kinerja model Multinomial Naïve Bayes dalam klasifikasi sentimen ulasan Alfagift yang direpresentasikan menggunakan pembobotan TF-IDF. Dataset penelitian terdiri dari 3.500 ulasan berbahasa Indonesia yang diperoleh melalui web scraping Google Play Store. Seluruh data diproses melalui tahapan NLP, mencakup cleaning, case folding, normalisasi, tokenisasi, stopword removal, dan stemming sebelum dilakukan ekstraksi fitur TF-IDF. Model dievaluasi menggunakan metrik akurasi, precision, recall, dan F1-score. Hasil penelitian menunjukkan bahwa sebelum penerapan SMOTE, model menghasilkan akurasi sebesar 0,8937 dengan recall sentimen positif hanya 0,77, menandakan bahwa model kurang mampu mengenali kelas minoritas. Setelah SMOTE diterapkan pada data latih, performa model meningkat menjadi akurasi 0,8967, dengan perbaikan signifikan pada kelas positif: precision mencapai 0,85, recall meningkat menjadi 0,88, dan F1-score naik menjadi 0,86. Perbaikan tersebut mengindikasikan bahwa teknik SMOTE berhasil menyeimbangkan distribusi data sehingga model lebih sensitif dalam mendeteksi sentimen minoritas tanpa mengorbankan stabilitas performa pada kelas mayoritas. Temuan ini membuktikan bahwa kombinasi preprocessing NLP, TF-IDF, Multinomial Naïve Bayes, dan SMOTE efektif digunakan untuk meningkatkan akurasi serta keseimbangan klasifikasi sentimen pada ulasan aplikasi Alfagift.

References

O. ; B. Bellar A.; Ballafkih M., “Sentiment analysis: Predicting product reviews for e-commerce recommendations using deep learning and transformers,” Mathematics, vol. 12, no. 15, p. 2403, 2024, doi: 10.3390/math12152403.

L. ; L. Yang Y.; Wang J.; Sherratt R. S., “Sentiment analysis for E-Commerce product reviews in Chinese based on sentiment lexicon and deep learning,” IEEE Access, vol. 8, pp. 23522–23530, 2020, doi: 10.1109/ACCESS.2020.2969854.

L. Huang, “Deep learning for text sentiment analysis: A survey,” Applied and Computational Engineering, vol. 104, pp. 135–139, 2024, doi: 10.54254/2755-2721/20241153.

C. Suhaeni and H.-S. Yong, “Mitigating class imbalance in sentiment analysis through GPT-3-generated synthetic sentences,” Applied Sciences, vol. 13, no. 17, p. 9766, 2023, doi: 10.3390/app13179766.

T. ; L. Kosolwattana C.; Hu R.; Han S.; Chen H.; Lin Y., “A self-inspected adaptive SMOTE algorithm (SASMOTE) for highly imbalanced data classification in healthcare,” BioData Min., vol. 16, p. Article 15, 2023, doi: 10.1186/s13040-023-00330-4.

M. Mukherjee and M. Khushi, “Smote-enc: A novel smote-based method to generate synthetic data for nominal and continuous features,” Applied System Innovation, vol. 4, no. 1, 2021, doi: 10.3390/asi4010018.

F. Duan, S. Zhang, Y. Yan, and Z. Cai, “An Oversampling Method of Unbalanced Data for Mechanical Fault Diagnosis Based on MeanRadius-SMOTE,” Sensors, vol. 22, no. 14, Jul. 2022, doi: 10.3390/s22145166.

J. H. Joloudari, A. Marefat, M. A. Nematollahi, S. S. Oyelere, and S. Hussain, “Effective class-imbalance learning based on SMOTE and convolutional neural networks,” Applied Sciences, vol. 13, no. 6, p. 4006, 2023, doi: 10.3390/app13064006.

F. Mujahid, B. Turkoglu, E. Kaya, and T. Asuroglu, “Data oversampling and imbalanced datasets: An investigation of performance for machine learning and feature engineering,” J. Big Data, vol. 11, p. Article 87, 2024, doi: 10.1186/s40537-024-00943-4.

B. Nemade, V. Bharadi, S. S. Alegavi, and B. Marakarkandy, “A comprehensive review: SMOTE-based oversampling methods for imbalanced classification techniques, evaluation, and result comparisons,” International Journal of Intelligent Systems and Applications in Engineering, vol. 11, no. 9s, pp. 790–803, 2023.

V. Gooljar, T. Issa, and S. Hardin-Ramanan, “Sentiment-based predictive models for online purchases in the era of Marketing 5.0: A systematic review,” J. Big Data, vol. 11, p. 107, 2024, doi: 10.1186/s40537-024-00947-0.

S. A. Hicks, T. Powo, and H. Booth, “On evaluation metrics for medical applications of artificial intelligence,” Sci. Rep., vol. 12, p. 6924, 2022, doi: 10.1038/s41598-022-09954-8.

M. Owusu-Adjei, E. Antwi, and J. Annan, “Imbalanced class distribution and performance evaluation metrics,” PLOS Digital Health, vol. 2, no. 7, p. e0000290, 2023, doi: 10.1371/journal.pdig.0000290.

W. I. Sabilla and C. B. Vista, “Implementation of SMOTE and under sampling on imbalanced datasets for predicting company bankruptcy,” Jurnal Komputer Terapan, vol. 7, no. 2, 2024, doi: 10.35143/jkt.v7i2.5027.

S. Gupta and G. Lehal, “A Survey on Natural Language Processing Techniques,” Information, vol. 12, no. 9, p. 345, 2021, doi: 10.3390/info12090345.

D. W. Otter, J. R. Medina, and J. K. Kalita, “A Survey of the Usages of Deep Learning for Natural Language Processing,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 2, pp. 604–624, 2020, doi: 10.1109/TNNLS.2020.2979670.

S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep Learning–Based Text Classification: A Comprehensive Review,” ACM Comput. Surv., vol. 54, no. 3, pp. 1–40, 2021, doi: 10.1145/3439726.

A. H. Putra and A. Salam, “A comparative performance of SMOTE, ADASYN and random oversampling in machine-learning models on prostate cancer dataset,” Journal of Applied Informatics and Computing, vol. 9, no. 3, 2024, doi: 10.30871/jaic.v9i3.9308.

T. Fulazzaky, A. Saefuddin, and A. M. Soleh, “Evaluating ensemble learning techniques for class imbalance in machine learning: A comparative analysis of Balanced Random Forest, SMOTE-RF, SMOTEBoost, and RUSBoost,” Scientific Journal of Informatics, vol. 11, no. 4, 2024, doi: 10.15294/sji.v11i4.15937.