Transfer Performance and LIME Explanation of Ensemble Classifiers in Cross-Project Defect Prediction
DOI:
https://doi.org/10.63158/journalisi.v8i3.1667Keywords:
SDP, Cross-project defect prediction, Ensemble Learning, SMOTE, LIME, ExplainabilityLeave-One-Project-Out, NASA MDAbstract
Ensemble methods are widely used in cross-project defect prediction (CPDP), particularly in projects that lack sufficient historical data by training on external source projects. However, no prior study has compared Bagging, Boosting, and Stacking directly under a Leave-One-Project-Out (LOPO) protocol or examined whether within-project performance rankings carry over to the cross-project setting. We evaluated three ensemble classifiers on five NASA MDP datasets sharing a common Halstead and McCabe feature schema. SMOTE is applied exclusively to pooled source data to prevent leakage into the target. A no-SMOTE baseline isolates the contribution of source-only SMOTE. LIME explanations are aggregated over thirty instances per model to assess feature importance consistency across the project boundary. Within-project evaluation shows Stacking achieves the highest F1 on four of five datasets, peaking at 0.503 on KC1. Under LOPO, these rankings reverse as Bagging and Boosting transfer more reliably, while Stacking's F1 drops by up to 0.258 points. Source-only SMOTE consistently improves transfer across all targets and ensembles. LIME consistency analysis produces undefined Spearman rank correlations, indicating that thirty-instance aggregation is insufficient to produce stable rank vectors for 21-feature datasets. To the best of our knowledge, this is the first study to compare all three ensemble strategies under LOPO on a shared NASA dataset feature schema. Particularly with a no-SMOTE control, aggregated LIME analysis, and a pilot meta-feature study identifying dataset size as the most actionable label-free predictor of ensemble suitability for CPDP deployment.
Downloads
References
[1] M. Ali, T. Mazhar, A. Al-Rasheed, T. Shahzad, Y. G. Yasin, and M. A. Khan, "Enhancing software defect prediction: A framework with improved feature selection and ensemble machine learning," PeerJ Comput. Sci., vol. 10, p. e1860, 2024, doi: 10.7717/peerj-cs.1860.
[2] M. Ali et al., "Software Defect Prediction Using an Intelligent Ensemble-Based Model," IEEE Access, vol. 12, pp. 20376–20395, 2024, doi: 10.1109/ACCESS.2024.3358201.
[3] B. Isong and E. Igo, "Ensemble Learning for Software Defect Prediction: Performance, Practicality and Future Directions," Journalisi, vol. 7, no. 3, pp. 2245–2291, Sep. 2025, doi: 10.51519/journalisi.v7i3.1171.
[4] X. Dong, Y. Liang, S. Miyamoto, and S. Yamaguchi, "Ensemble learning based software defect prediction," J. Eng. Res., vol. 11, no. 4, pp. 377–391, 2023.
[5] D. Al-Fraihat, Y. Sharrab, A.-R. Al-Ghuwairi, H. Alshishani, and A. Algarni, "Hyperparameter Optimization for Software Bug Prediction Using Ensemble Learning," IEEE Access, vol. 12, pp. 51869–51878, 2024, doi: 10.1109/ACCESS.2024.3380024.
[6] A. Vescan, R. Găceanu, and C. Şerban, "Exploring the impact of data preprocessing techniques on composite classifier algorithms in cross-project defect prediction," Autom. Softw. Eng., vol. 31, p. 47, 2024, doi: 10.1007/s10515-024-00454-9.
[7] N. Nikravesh and M. R. Keyvanpour, "Cross-project Defect Prediction with an Enhanced Transfer Boosting Algorithm," in Proc. 12th Int. Conf. Comput. Knowl. Eng. (ICCKE), Mashhad, Iran, 2022, pp. 157–162, doi: 10.1109/ICCKE57176.2022.9960103.
[8] T. Asano et al., "Using Bandit Algorithms for Project Selection in Cross-Project Defect Prediction," in Proc. IEEE Int. Conf. Softw. Maintenance Evol. (ICSME), Luxembourg, 2021, pp. 649–653, doi: 10.1109/ICSME52107.2021.00074.
[9] S. Zheng, J. Gai, H. Yu, H. Zou, and S. Gao, "Training data selection for imbalanced cross-project defect prediction," Comput. Electr. Eng., vol. 94, p. 107370, 2021.
[10] J. Chen, J. Ding, K. C. Tan, J. Qian, and K. Li, "MBL-CPDP: A Multi-Objective Bilevel Method for Cross-Project Defect Prediction," IEEE Trans. Softw. Eng., vol. 51, no. 8, pp. 2305–2328, Aug. 2025, doi: 10.1109/TSE.2025.3577808.
[11] H. Tong et al., "MASTER: Multi-Source Transfer Weighted Ensemble Learning for Multiple Sources Cross-Project Defect Prediction," IEEE Trans. Softw. Eng., vol. 50, no. 5, pp. 1281–1305, May 2024, doi: 10.1109/TSE.2024.3381235.
[12] H. Tong, W. Lu, W. Xing, and S. Wang, "ARRAY: Adaptive triple feature-weighted transfer Naive Bayes for cross-project defect prediction," J. Syst. Softw., vol. 202, p. 111721, 2023.
[13] O. P. Omondiagbe, S. A. Licorish, and S. G. MacDonell, "Improving transfer learning for software cross-project defect prediction," Appl. Intell., vol. 54, pp. 5593–5616, 2024.
[14] J. Chen, J. Xu, S. Cai, X. Wang, H. Chen, and Z. Li, "Software Defect Prediction Approach Based on a Diversity Ensemble Combined With Neural Network," IEEE Trans. Rel., vol. 73, no. 3, pp. 1487-1501, Sept. 2024, doi: 10.1109/TR.2024.3356515.
[15] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic Minority Over-sampling Technique," J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002.
[16] X. Fan, S. Zhang, K. Wu, W. Zheng, and Y. Ge, "Cross-Project Software Defect Prediction Based on SMOTE and Deep Canonical Correlation Analysis," Comput. Mater. Contin., vol. 78, pp. 1687–1711, 2024.
[17] M. Mustaqeem, T. Siddiqui, and S. Mustajab, "A hybrid-ensemble model for software defect prediction for balanced and imbalanced datasets using AI-based techniques with feature preservation: SMERKP-XGB," J. Softw. Evol. Proc., vol. 37, no. 1, p. e2731, Jan. 2025, doi: 10.1002/smr. 2731.
[18] M. Shepperd, Q. Song, Z. Sun, and C. Mair, "Data Quality: Some Comments on the NASA Software Defect Datasets," IEEE Trans. Softw. Eng., vol. 39, no. 9, pp. 1208–1215, Sept. 2013, doi: 10.1109/TSE.2013.11.
[19] T. Menzies, J. Greenwald, and A. Frank, "Data Mining Static Code Attributes to Learn Defect Predictors," IEEE Trans. Softw. Eng., vol. 33, no. 1, pp. 2–13, Jan. 2007, doi: 10.1109/TSE.2007.256941.
[20] R. Haque, A. Ali, S. McClean, I. Cleland, and J. Noppen, "Heterogeneous Cross-Project Defect Prediction Using Encoder Networks and Transfer Learning," IEEE Access, vol. 12, pp. 409-419, 2024, doi: 10.1109/ACCESS.2023.3343329.
[21] W. Wang, Y. Li, S. Song, J. Lu, B. Chen, and B. Wang, "Research on Cross-project Defect Prediction Based on Instance Migration Method," in Proc. 3rd Int. Conf. Comput. Sci. Manage. Technol. (ICCSMT), Shanghai, China, 2022, pp. 300–303, doi: 10.1109/ICCSMT58129.2022.00070.
[22] Z. Sun, J. Li, H. Sun, and L. He, "CFPS: Collaborative filtering based source projects selection for cross-project defect prediction," Appl. Soft Comput., vol. 99, p. 106940, 2021.
[23] M. Patil, M. Bisi, and P. Manchala, "Source Project Selection for Cross-Project Software Defect Prediction using Clustering Approach," in Proc. IEEE 20th India Council Int. Conf. (INDICON), Hyderabad, India, 2023, pp. 904–909, doi: 10.1109/INDICON59947.2023.10440956.
[24] C. Jin, "Cross-project software defect prediction based on domain adaptation learning and optimization," Expert Syst. Appl., vol. 171, p. 114637, Jun. 2021, doi: 10.1016/j.eswa.2021.114637.
[25] Y. Khatri and U. R. Saxena, "Dynamic learner selection for cross-project fault prediction," Int. J. Syst. Assur. Eng. Manage., vol. 16, pp. 532–551, 2025, doi: 10.1007/s13198-024-02586-3.
[26] A. M. Ibrahim, H. Abdelsalam, and I. A. T. F. Taj-Eddin, "Software Defects Prediction At Method Level Using Ensemble Learning Techniques," Int. J. Intell. Comput. Inf. Sci., vol. 23, no. 2, pp. 28-49, 2023, doi: 10.21608/ijicis. 2023.189934.1251.
[27] G. Lee, H. Ju, and S. U.-J. Lee, "Can We Trust the Actionable Guidance from Explainable AI Techniques in Defect Prediction? " in Proc. IEEE Int. Conf. Softw. Anal. Evol. Reeng. (SANER), Montreal, QC, Canada, 2025, pp. 476–487, doi: 10.1109/SANER64311.2025.00051.
[28] E. A. Felix and S. P. Lee, "Predicting the number of defects in a new software version," PLoS ONE, vol. 15, no. 3, Art. no. e0229131, Mar. 2020, doi: 10.1371/journal.pone.0229131.
[29] Rashmi and A. Kaur, "Smadasyn-boosted cross-project transfer learning for effective security fault prediction," Evol. Intell., vol. 18, p. 76, 2025, doi: 10.1007/s12065-025-01060-8.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Information Systems and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors Declaration
- The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
- The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
- The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
- The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
- The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
- The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
- If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
- The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights
