Comparative Evaluation of BiLSTM-CNN, XGBoost, and Ridge Regression for Heart Disease Classification on the Cleveland Dataset
DOI:
https://doi.org/10.63158/journalisi.v8i3.1668Keywords:
BiLSTM-CNN, Random Forest, heart disease classification, imbalanced learning, AUC, Small MedicalAbstract
Transformers have become the dominant architecture for tabular data modelling in natural language processing; however, their effectiveness for numerical tabular classification on modest sized and moderately imbalanced datasets remains unclear. This study evaluates the performance of hybrid deep learning and classical machine learning models which use the Cleveland Heart Disease dataset with 297 complete observations and was artificially constructed from 13 clinical features. The models examined include BiLSTM-CNN, Random Forest, XGBoost, Logistic Regression, and Ridge Regression. An experimental comparative approach was adopted under identical preprocessing, training conditions, and evaluation metrics, including accuracy, recall, F1-score, and Area Under the Curve (AUC). Results show that BiLSTM-CNN achieved the highest recall (0.8478), demonstrating strong minority class detection capability. Random Forest and XGBoost produced the best-balanced performance with 81.67% accuracy and the BiLSTM-CNN has the best F1-score of 0.8364, while Ridge Regression achieved the highest AUC (0.8945). This study provides empirical evidence that hybrid recurrent and ensemble models perform optimally on a small to medium sized Cleveland Heart Desease numerical tabular datasets without pre-training, offering practical guidance for Cleveland Heart Disease tabular clinical classification tasks, and no external validation was performed.
Downloads
References
[1] G. P. Reddy, S. Member, and P. D. Javali, “A Hybrid Deep Learning Approach With Explainable AI for Diabetic Retinopathy Classification,” IEEE Access, vol. 14, no. February, pp. 28819–28839, 2026, doi: 10.1109/ACCESS.2026.3665564.
[2] N. Habashneh, M. Alwars, M. Alshehhi, and M. A. Al-Betar, “Comparative Analysis of Machine Learning Models for Heart Disease Prediction Using the Cleveland Dataset,” in 2025 10th International Conference on Information Technology Trends (ITT), 2025, pp. 158–163. doi: 10.1109/ITT69610.2025.11352907.
[3] S. Hariharan, Y. A. Jerusha, G. Suganeshwari, S. P. S. Ibrahim, U. Tupakula, and V. Varadharajan, “A Hybrid Deep Learning Model for Network Intrusion Detection System Using Seq2Seq and ConvLSTM-Subnets,” IEEE Access, vol. 13, no. January, pp. 30705–30721, 2025, doi: 10.1109/ACCESS.2025.3541399.
[4] V. Sivaprasad, M. Rahmati, S. Member, A. Springer, and H. Vereecken, “Development of Continuous AMSR-E / 2 Soil Moisture Time Series by Hybrid Deep Learning Model ( ConvLSTM2D and Conv2D ) and Transfer Learning for Reanalyses,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 18, pp. 11169–11182, 2025, doi: 10.1109/JSTARS.2025.3557956.
[5] A. Abdulkareem, H. Brihi, and M. Ghurab, “A Hybrid Deep Learning-Machine Learning Stacking Model for Yemeni Arabic Dialect Sentiment Analysis,” IEEE Access, vol. 13, no. July, pp. 134160–134171, 2025, doi: 10.1109/ACCESS.2025.3593204.
[6] N. K. Al-qazzaz et al., “Transfer Learning and Hybrid Deep Convolutional Neural Networks Models for Autism Spectrum Disorder Classification From EEG Signals,” IEEE Access, vol. 12, no. March, pp. 64510–64530, 2024, doi: 10.1109/ACCESS.2024.3396869.
[7] A. Jabbar et al., “A Lesion-Based Diabetic Retinopathy Detection Through Hybrid Deep Learning Model,” no. March, 2024, doi: 10.1109/ACCESS.2024.3373467.
[8] J. Ali et al., “A Hybrid Deep Learning Model to Predict High-Risk Students in Virtual Learning Environments,” no. July, pp. 103687–103703, 2024.
[9] E. N. Casmiry, R. S. Sinde, and N. M. Mduma, “A Hybrid Deep Learning Model for SQL Injection Attack Detection,” IEEE Access, vol. 14, no. December 2025, pp. 6450–6463, 2026, doi: 10.1109/ACCESS.2026.3651991.
[10] A. Kala, O. Torkul, and I. H. Selvi, “Early Prediction of Student Performance in Face-to-Face Education Environments : A Hybrid Deep Learning Approach With XAI Techniques,” IEEE Access, vol. 12, no. December, pp. 191635–191649, 2024, doi: 10.1109/ACCESS.2024.3516816.
[11] C. Hsiao et al., “Precision and Robust Models on Healthcare Institution Federated Learning for Predicting HCC on Portal Venous CT Images,” IEEE J. Biomed. Heal. Informatics, vol. 28, no. 8, pp. 4674–4687, 2024, doi: 10.1109/JBHI.2024.3400599.
[12] D. Kumar and A. K. Gupta, “Integrating Data Augmentation and Meta-Heuristic Optimization Algorithms for Enhanced Hybrid Nanofluid Density Prediction Through Machine and Deep Learning Paradigms,” IEEE Access, vol. 13, no. January, pp. 35750–35779, 2025, doi: 10.1109/ACCESS.2025.3543475.
[13] M. TRIPTI, J. HELEN, S. RASHMI, P. POORVI, A. ABEER, and V. PRAKASH, “Deep vs . Shallow : A Comparative Study of Machine Learning and Deep Learning Approaches for Fake Health News Detection,” IEEE Access, vol. 11, no. August, 2023.
[14] K. Zaman, M. Sah, C. Direkoglu, and M. Unoki, “A Survey of Audio Classification Using Deep Learning,” IEEE Access, vol. 11, no. October, pp. 106620–106649, 2023, doi: 10.1109/ACCESS.2023.3318015.
[15] S. Roy, “Two-Stage Hybrid Deep Learning Architecture for Cross-Domain Anomaly Detection and,” IEEE Open J. Comput. Soc., vol. 7, no. December 2025, pp. 117–128, 2026, doi: 10.1109/OJCS.2025.3643329.
[16] S. MAYANDA MEGA, B. T., J. KASIYAH, and L. OENARDI, “Automatic Detection of Students ’ Engagement During Online Learning : A Bagging Ensemble Deep Learning Approach,” vol. 12, no. July, pp. 96063–96073, 2024.
[17] C. G. Onyiagha, G. F. Yanwalo, and N. E. Ajimah, “Phishing URL Detection: A Basic Machine Learning Approach,” Int. J. Sci. TECHNOLEDGE, vol. 12, no. 3, pp. 8–14, 2024, doi: 10.24940/theijst/2024/v12/i3/st2403-001.
[18] M. Hamza et al., “New Hybrid Deep Learning Models to Predict Cost From Healthcare Providers in Smart Hospitals,” IEEE Access, vol. 11, no. August, pp. 136988–137010, 2023, doi: 10.1109/ACCESS.2023.3336424.
[19] M. A. Kadhim and A. M. Radhi, “Heart disease classification using optimized Machine learning algorithms Heart disease classification using optimized Machine learning algorithms,” Iraqi J. Comput. Sci. Math., vol. 4, no. 2, pp. 31–42, 2023, doi: 10.52866/ijcsm.2023.02.02.004.
[20] P. H. Hutagalung, S. Informasi, and U. Nasional, “Heart Disease Classification Using Optimised XGBoost and Random Forest with SHAP Explanations,” Sink. J. dan Penelit. Tek. Inform., vol. 10, no. 1, pp. 330–342, 2026, doi: 10.33395/sinkron.v10i1.15544 e-ISSN.
[21] F. Gemci, “A Novel Comparative Approach : Logistic Regression Enhanced by Bat Optimization Versus Logistic Regression Enhanced by Deep Belief Network for Remote Homologous Protein Detection,” IEEE Access, vol. 13, no. November, pp. 209723–209728, 2025, doi: 10.1109/ACCESS.2025.3641298.
[22] B. Zhang and C. Peng, “Robust Sparse Logistic Regression With the L q ( 0 < q < 1 ) Regularization for Feature Selection Using Gene Expression Data,” vol. 6, 2018, doi: 10.1109/ACCESS.2018.2880198.
[23] U. Irvine, “Cleveland Heart Disease dataset,” Dataset. Accessed: Apr. 04, 2026. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/Heart+Disease
[24] Z. Shangli, Z. Lili, Q. I. U. Kuanmin, L. U. Ying, and C. A. I. Baigen, “Variable Selection in Logistic Regression Model ∗,” vol. 24, no. 4, 2015, doi: 10.1049/cje.2015.10.025.
[25] N. Fazakis, S. Kotsiantis, and Y. Dimakopoulos, “MAGA : A Controlled Minority-Class Generative-Augmentation Framework With Fairness-Aware Selection for Imbalanced Diabetes Tabular Classification,” IEEE Access, vol. 14, no. March 2026, 2026.
[26] Y. Chen, S.-S. LIN, Y. SHI, T.-Y. HO, and X. XU, “MCC : Multi-Cluster Contrastive Semi-Supervised Segmentation Framework for Echocardiogram Videos,” IEEE Access, vol. 13, no. February, pp. 30543–30554, 2025, doi: 10.1109/ACCESS.2025.3541173.
[27] L. I. U. Chunhui, Q. I. Yue, and D. Wenrui, “The Data-Reusing MCC-Based Algorithm and Its Performance Analysis ∗,” vol. 25, no. 4, 2016, doi: 10.1049/cje.2016.06.019.
[28] R. Matsuo, S. Yasuda, and H. Yoshida, “Multiple Instance Learning With Instance-Level Positive-Unlabeled Learning in Anomaly Detection,” IEEE Access, vol. 13, no. April, pp. 103627–103639, 2025, doi: 10.1109/ACCESS.2025.3578651.
[29] S. K. Patel, S. Member, J. Surve, and G. S. Member, “Encoding and Tuning of THz Metasurface-Based Refractive Index Sensor With Behavior Prediction Using XGBoost Regressor,” IEEE Access, vol. 10, pp. 24797–24814, 2022, doi: 10.1109/ACCESS.2022.3154386.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
