افتح

التعلم الجماعي مع الذكاء الاصطناعي القابل للتفسير لتحسين توقعات أمراض القلب استنادًا إلى مجموعات بيانات متعددة

• تصميم نماذج التكديس والتصويت لتوقع أمراض القلب: نقدم إطار عمل شاملاً يجمع بين خوارزميات تعلم الآلة المتنوعة لتعزيز الأداء التنبؤي.
• معالجة اختيار نماذج أساسية متنوعة: نحن نعتبر مجموعة من خوارزميات التعلم الآلي التي تتيح لنماذجنا التقاط نقاط القوة في أساليب مختلفة والتخفيف من نقاط ضعفها.
• إجراء تجارب على مجموعات بيانات متعددة: تم إجراء تجاربنا الدقيقة على مجموعتين متميزتين من بيانات أمراض القلب.
• تقييم شامل للنماذج: يتم تقييم النماذج المجمعة المقترحة بدقة باستخدام مقاييس متنوعة، مما يظهر تفوقها على النماذج الأساسية الفردية والنماذج المتطورة.
• تطبيق تحليل إحصائي صارم: لضمان الأهمية الإحصائية لتحسينات الأداء، يتم تنفيذ إطار إحصائي قوي – بما في ذلك اختبار رتب فريدمان المتوافقة وتحليل هولم بعد الاختبار.
• دمج الذكاء الاصطناعي القابل للتفسير (XAI) من خلال SHAP: تدمج دراستنا تقنيات XAI، وبشكل خاص SHAP (تفسيرات شابلي الإضافية)، لتفسير التنبؤات التي تقوم بها نماذج التجميع والتصويت. وهذا يمكننا من توفير الشفافية في تنبؤات النموذج وفهم أفضل لكيفية تأثير الميزات المختلفة على القرار النهائي، مما يعالج التحديات المتعلقة بالتفسير التي غالبًا ما ترتبط بالنماذج المركبة المعقدة.

الأعمال ذات الصلة

• قمنا باستكشاف شامل لمجموعة متنوعة من النماذج الأساسية ذات الخصائص المختلفة لتطوير أطر العمل الخاصة بالتكديس والتصويت.
• قمنا بتصميم خطوط أنابيب فريدة لتعزيز الفعالية والعمومية والصلابة لهذه الأطر الخاصة بالتكديس والتصويت.
• قمنا بدراسة دور التكديس والتصويت في تقديم رؤى قيمة حول الميزات والنماذج الأساسية التي تؤثر على التنبؤ النهائي، مما يعزز فهمًا أفضل للمرض.
• استخدمنا اختبار رتب فريدمان المتوافقة مع تحليل هولم اللاحق لتأكيد الدلالة الإحصائية لأداء النموذج المصمم.
• على عكس العديد من الدراسات السابقة التي تعتبر نماذج التجميع كـ “صناديق سوداء”، يدمج عملنا الذكاء الاصطناعي القابل للتفسير لتوفير الشفافية في توقعات النموذج. وهذا يميز دراستنا من خلال تقديم رؤى قابلة للتفسير حول كيفية تأثير الميزات الفردية على التوقع النهائي، مما يعالج القضية التي غالبًا ما يتم تجاهلها وهي قابلية تفسير النموذج في توقع أمراض القلب.

منهجية البحث

سير عمل البحث

التكديس والتصويت

الشكل 1. المنهجية المقترحة للبحث.

الشكل 2. مزايا التكديس.

الشكل 3. مزايا التصويت.

تكديس

التصويت

نماذج المكونات

نماذج ضعيفة

نماذج التجميع

مجموعة البيانات للتجربة

الشكل 4. توزيع المتغيرات المستهدفة في كلا المجموعتين البيانيّتين.

التجربة والنتائج

مقاييس التقييم

• الإيجابي الحقيقي (TP) يعني أن المريض يعاني من مرض القلب، وأن النموذج يتنبأ بذلك بشكل صحيح.
• السلبية الحقيقية (TN) تعني أن المريض لا يعاني من مرض القلب، وأن النموذج يتنبأ بذلك بدقة.
• تشير الإيجابيات الكاذبة (FPs) إلى أن المريض لا يعاني من مرض القلب، ومع ذلك يتنبأ النموذج بشكل غير صحيح بنتيجة إيجابية لمرض القلب.
• يمثل السلبية الكاذبة (FN) حالة يكون فيها المريض مصابًا بأمراض القلب، لكن النموذج يتنبأ بشكل غير صحيح بنتيجة سلبية.

نتائج التنبؤ للنماذج الأساسية

تصميم خط الأنابيب للتكديس والتصويت

الشكل 7. أداء النماذج الأساسية مع D1.

الشكل 8. أداء النماذج الأساسية مع D2.

الشكل 9. اختيار النموذج.

الشكل 10. بناء خط الأنابيب للتكديس والتصويت.

الشكل 11. الأداء المقارن لـ LR و KNN و LDA كمتعلمين ميتا.

Input: a) Training/validation dataset $boldsymbol{T}_{boldsymbol{R}}=left{boldsymbol{x}_{boldsymbol{i}}, boldsymbol{y}_{boldsymbol{i}}right}_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}}, boldsymbol{n}$ is no. of instances
    b) Base models: $boldsymbol{B}_{boldsymbol{L}}=left{boldsymbol{b}_{boldsymbol{1}}+boldsymbol{b}_{boldsymbol{2}}+ldots+boldsymbol{b}_{boldsymbol{k}}right}$
    c) A meta-classifier (LR)
Output: Ensemble stacking classifier S
    Initialize an empty 2D array $boldsymbol{S}$ of size $n times k$ to store base learner predictions
    For each base learner $boldsymbol{b}_{boldsymbol{j}}$ in $boldsymbol{B}_{boldsymbol{L}}$ do
        Initialize hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Initialize an empty 1D array $boldsymbol{P}$ to store validation metrics $boldsymbol{P}_{i}$
        For each instance $boldsymbol{i} in{1,2,3, ldots, n}$
            Train $boldsymbol{b}_{boldsymbol{j}}$ on $boldsymbol{T}_{boldsymbol{R}}^{(-boldsymbol{i})}$ // Train all instances except $i^{text {th }}$; keep one for cross-validation
            Predict $boldsymbol{x}_{boldsymbol{i}}$ and compute performance metric $boldsymbol{P}_{boldsymbol{i}}$
            Append $boldsymbol{P}_{boldsymbol{i}}$ to $boldsymbol{P}$
        End
        Compute average validation score $boldsymbol{P}=frac{mathbf{1}}{boldsymbol{n}} sum_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}} boldsymbol{P}_{boldsymbol{i}} / /$ Aggregate performance
        Adjust hyperparameters to maximize $boldsymbol{P}$ // Optimize hyperparameters
        Go to Step 4 until $boldsymbol{P}$ improvement $<1 %$ || Saturation
        Store the best hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Train $boldsymbol{b}_{boldsymbol{j}}$ on full $boldsymbol{T}_{boldsymbol{R}} / /$ Final training
        Perform cross-validation using the remaining instance
        Store the predicted class labels for $boldsymbol{T}_{boldsymbol{R}}$ in column $boldsymbol{j}$ of $boldsymbol{S}$
    End
    Initialize another $2 D$ array $boldsymbol{T}_{boldsymbol{N}}$ of size $n times(k+1) / /$ Prepare meta-dataset
    Concatenate $boldsymbol{S}$ with original labels $boldsymbol{y}: boldsymbol{T}_{boldsymbol{N}}=[boldsymbol{S} boldsymbol{y}]$
    For each instance $boldsymbol{i} in{1,2,3, ldots, n} / /$ Train meta-learner (LR)
        Train $boldsymbol{L} boldsymbol{R}$ on $boldsymbol{T}_{boldsymbol{N}}^{(-boldsymbol{i})}$ // Train all instances except $i^{text {th }}$; keep one for cross-validation
        Predict $boldsymbol{x}_{boldsymbol{i}}$ and store the result in $boldsymbol{S}$
    End
    Return the predicted class labels in $boldsymbol{S}$

Input: a) Training/validation dataset $boldsymbol{T}_{boldsymbol{R}}=left{boldsymbol{x}_{boldsymbol{i}}, boldsymbol{y}_{boldsymbol{i}}right}_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}}, boldsymbol{n}$ is no. of instances
        b) Base models: $boldsymbol{B}_{L}=left{boldsymbol{b}_{1}+boldsymbol{b}_{2}+ldots+boldsymbol{b}_{k}right}$
Output: Ensemble voting classifier $boldsymbol{V}$
    Initialize an empty 2D array $boldsymbol{V}$ of size $n times k$ to store base learner predictions
    Initialize an empty 2D array $boldsymbol{C}$ to store confidence scores for each prediction
    For each base learner $boldsymbol{b}_{boldsymbol{j}}$ in $boldsymbol{B}_{boldsymbol{L}}$ do
        Initialize hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Initialize an empty 1D array $boldsymbol{P}$ to store validation metrics $boldsymbol{P}_{boldsymbol{i}}$
        For each instance $boldsymbol{i} in{1,2,3, ldots, n}$
            Train $boldsymbol{b}_{boldsymbol{j}}$ on $boldsymbol{T}_{boldsymbol{R}}^{(-boldsymbol{i})}$ // Train all instances except $i^{text {th }}$; keep one for cross-validation
            Predict $boldsymbol{x}_{boldsymbol{i}}$ and compute performance metric $boldsymbol{P}_{boldsymbol{i}}$
            Append $boldsymbol{P}_{boldsymbol{i}}$ to $boldsymbol{P}$
        End
        Compute the average validation score $boldsymbol{P}=frac{mathbf{1}}{boldsymbol{n}} sum_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}} boldsymbol{P}_{boldsymbol{i}} / /$ Aggregate performance
        Adjust hyperparameters to maximize $boldsymbol{P}$ // Optimize hyperparameters
        Go to Step 4 until $boldsymbol{P}$ improvement < $1 %$ // Saturation
        Store the best hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Train $boldsymbol{b}_{boldsymbol{j}}$ on full $boldsymbol{T}_{boldsymbol{R}} / /$ Final training
        Perform cross-validation using the remaining instance
    Store the predicted class labels for $boldsymbol{T}_{R}$ in column $boldsymbol{j}$ of $boldsymbol{V}$ and confidence scores in $boldsymbol{C}$
    End
    Calculate the weighted vote for each class label based on the confidence scores in $boldsymbol{C}$
    Assign the class label with the highest weighted vote to $V$ for this instance
    Return the predicted class labels in $V$

التحقق المتقاطع

تقييم أهمية الميزات

الشكل 12. عملية التحقق المتقاطع -fold.

الشكل 13. أهمية الميزات لـ (أ) التكديس مع D1، (ب) التصويت مع D1، (ج) التكديس مع D2 و (د) التصويت مع D2.

تعديل المعلمات الفائقة

نتائج التنبؤ لنماذج التجميع والتصويت

التحليل والمناقشة

نماذج التجميع والتصويت مقارنة بالنماذج الأساسية

التحليل الإحصائي لنماذج التجميع والتصويت

الشكل 14. مصفوفات الالتباس لـ (أ) التكديس مع D1، (ب) التصويت مع D1، (ج) التكديس مع D2 و (د) التصويت مع D2.

الشكل 15. دقة التجميع والتصويت بعشرة أضعاف على كلا المجموعتين البيانيّتين.

الشكل 16. مقاييس أداء أخرى للتكديس والتصويت على كلا المجموعتين البيانيّتين.

الشكل 17. الانحراف المعياري للطيّات لمقاييس مختلفة للتكديس والتصويت مع كلا المجموعتين البيانيّتين.

تفسير نماذج التكديس والتصويت باستخدام SHAP

الشكل 18. ROC-AUC لـ (أ) التكديس مع D1، (ب) التصويت مع D1، (ج) التكديس مع D2 و (د) التصويت مع D2.

تفسير عالمي

الشكل 19. AUPRC لـ (أ) التكديس مع D1، (ب) التصويت مع D1، (ج) التكديس مع D2 و (د) التصويت مع D2.

الشكل 20. MCR للتكديس والتصويت مع كلا المجموعتين البيانيّتين.

تفسير محلي

الشكل 21. وقت التنفيذ (بالثواني) للتكديس والتصويت على كلا المجموعتين البيانيّتين.

الشكل 22. مقارنة نماذج التكديس والتصويت مع نماذج الأداء الأعلى من حيث (أ) الدقة، (ب) الدقة الإيجابية، (ج) الاسترجاع، (د) درجة F1، و(هـ) ROC مع كلا المجموعتين البيانيات.

الشكل 23. متوسط SHAP المطلق لـ (أ) التجميع و (ب) التصويت على D1.

الشكل 24. متوسط SHAP المطلق لـ (أ) التجميع و (ب) التصويت على D2.

الشكل 25. مخطط الشلال للتجميع على D1.

الشكل 26. مخطط الشلال للتصويت على D1.

الشكل 27. مخطط الشلال للتجميع على D2.

الشكل 28. مخطط الشلال للتصويت على D2.

الشكل 29. مخطط القوة لـ (أ) التجميع و (ب) التصويت على D1.

الشكل 30. مخطط القوة لـ (أ) التجميع و (ب) التصويت على D2.

مقارنة نماذج التكديس والتصويت مع أحدث التقنيات

الاستنتاجات، القيود، والاتجاهات المستقبلية

توفر البيانات

References

1. WHO. Cardiovascular diseases (CVDs), 11 June 2021. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/car diovascular-diseases-(cvds). [Accessed 17 December 2023].
2. Ahmad, G. N., Ullah, S., Algethami, A., Fatima, H. & Akhter, S. M. H. Comparative study of optimum medical diagnosis of human heart disease using machine learning technique with and without sequential feature selection. IEEE Access. 10, 23808-23828 (2022).
3. Gheorghe, A. et al. The economic burden of cardiovascular disease and hypertension in low- and middle-income countries: a systematic review. BMC Public. Health. 18, 975 (2018). (Article number.
4. Ruan, Y. et al. Cardiovascular disease (CVD) and associated risk factors among older adults in six low-and middle-income countries: results from SAGE wave 1. BMC Public. Health. 18(1), 1-13 (2018).
5. Biglu, M. H., Ghavami, M. & Biglu, S. Cardiovascular diseases in the mirror of science. J. Cardiovasc. Thorac. Res. 8(4), 158-163 (2016).
6. Ayano, Y. M., Schwenker, F., Dufera, B. D. & Debelee, T. G. Interpretable Machine Learning Techniques in ECG-Based Heart Disease Classification: A Systematic Review, Diagnostics, vol. 13, no. 1, p. 111, (2023).
7. Rath, A., Mishra, D., Panda, G. & Satapathy, S. C. An exhaustive review of machine and deep learning based diagnosis of heart diseases. Multimedia Tools Appl. 81, 36069-36127 (2022).
8. Ganie, S. M., Pramanik, P. K. D., Malik, M. B., Nayyar, A. & Kwak, K. S. An improved ensemble learning approach for heart disease prediction using boosting algorithms. Comput. Syst. Sci. Eng. 46(3), 3993-4006 (2023).
9. Brown, G. Ensemble learning, in Encyclopedia of Machine Learning, (eds Sammut, C. & Webb, G. I.) Boston, MA, Springer, 312-320. (2011).
10. Ganie, S. M. & Malik, M. B. An ensemble machine learning approach for predicting type-II diabetes mellitus based on lifestyle indicators. Healthc. Analytics. 22, 100092 (2022). (Article number.
11. Naveen, R. K., Sharma & Nair, A. R. Efficient breast cancer prediction using ensemble machine learning models, in 4 th International Conference on Recent Trends on Electronics, Information, Communication & Technology (RTEICT), Bangalore, India, (2019).
12. Oswald, G. J., Sathwika & Bhattacharya, A. Prediction of cardiovascular disease (CVD) using ensemble learning algorithms, in 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD), Bangalore, India, (2022).
13. Shanbhag, G. A., Prabhu, K. A., Subba Reddy, N. V. & Rao, B. A. Prediction of lung cancer using ensemble classifiers, Journal of Physics: Conference Series, vol. 2161 (012007), (2022).
14. Verma, A. K., Pal, S. & Tiwari, B. B. Skin disease prediction using ensemble methods and a new hybrid feature selection technique. Iran. J. Comput. Sci. 3, 207-216(2020).
15. Ganie, S. M. & Malik, M. B. Comparative analysis of various supervised machine learning algorithms for the early prediction of type-II diabetes mellitus. Int. J. Med. Eng. Inf. 14(6), 473-483 (2022).
16. Shaikh, F. J. & Rao, D. S. Prediction of cancer disease using machine learning approach, Materialstoday: Proceedings, vol. 50 (Part 1), pp. 40-47, (2022).
17. Senthilkumar, B. et al. Ensemble modelling for early breast cancer prediction from diet and lifestyle, IFAC-PapersOnLine, vol. 55, no. 1, pp. 429-435, (2022).
18. Ganie, S. M. & Pramanik, P. K. D. Predicting chronic liver disease using boosting, in 1st International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI-2023), Raipur, India, (2024).
19. Ganie, S. M., Pramanik, P. K. D., Mallik, S. & Zhao, Z. Chronic kidney disease prediction using boosting techniques based on clinical parameters. PLoS ONE. 18(12), e0295234 (2023).
20. Ganie, S. M., Pramanik, P. K. D., Malik, M. B., Mallik, S. & Qin, H. An ensemble learning approach for diabetes prediction using boosting techniques. Front. Genet. 14 (2023).
21. Ganie, S. M. & Pramanik, P. K. D. A comparative analysis of boosting algorithms for chronic liver disease prediction. Healthc. Analytics 5, 100313 (2024).
22. Shaik, H. S., RajyaLakshmi, G. V., Alane, V. & Kandimalla, N. D. Enhancing prediction of cardiovascular disease using bagging technique. in International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT).
23. Yuan, X. et al. A High accuracy integrated bagging-fuzzy-GBDT prediction algorithm for heart disease diagnosis. in IEEE/CIC International Conference on Communications in China (ICCC)(2019).
24. Deshmukh, V. M. Heart disease prediction using ensemble methods. Int. J. Recent. Technol. Eng. 8(3), 8521-8526 (2019).
25. Mary, N. et al. Investigating of classification algorithms for heart disease risk prediction. J. Intell. Med. Healthc. 1(1), 11-31 (2022).
26. Budholiya, K., Shrivastava, S. K. & Sharma, V. An optimized XGBoost based diagnostic system for effective prediction of heart disease. J. King Saud Univ. – Comput. Inform. Sci. 34(7), 4514-4523 (2022).
27. Pan, C., Poddar, A., Mukherjee, R. & Ray, A. K. Impact of categorical and numerical features in ensemble machine learning frameworks for heart disease prediction. Biomed. Signal Process. Control. 76, 103666 (2022).
28. Pouriyeh, S. et al. A comprehensive investigation and comparison of machine learning techniques in the domain of heart disease. in IEEE Symposium on Computers and Communications (ISCC) (2017).
29. Shorewala, V. Early detection of coronary heart disease using ensemble techniques. Inf. Med. Unlocked, 26, 100655 (2021).
30. Latha, C. B. C. & Jeeva, S. C. Improving the accuracy of prediction of heart disease risk based on ensemble classification techniques. Inf. Med. Unlocked 16, 100203 (2019).
31. Chandrasekhar, N. & Peddakrishna, S. Enhancing heart disease prediction accuracy through machine learning techniques and optimization. Processes 11, (4), 1210 (2023).
32. Tiwari, A., Chugh, A. & Sharma, A. Ensemble framework for cardiovascular disease prediction. Comput. Biol. Med. 146, 105624 (2022).
33. Raza, K. Improving the prediction accuracy of heart disease with ensemble learning and majority voting rule. in U-Healthcare Monitoring Systems, Design and Applications (eds Dey, N. et al.) (Academic, 2019).
34. Mienye, I. D., Sun, Y. & Wang, Z. An improved ensemble learning approach for the prediction of heart disease risk. Inf. Med. Unlocked. 20, 100402 (2020).
35. Ambrews, A. B. et al. Ensemble based machine learning model for heart disease prediction. in International Conference on Communications, Information, Electronic and Energy Systems (CIEES) (2022).
36. Ashfaq, A. et al. Multi-model ensemble based approach for heart disease diagnosis. in International Conference on Recent Advances in Electrical Engineering & Computer Sciences (RAEE & CS) (2022).
37. Habib, A. Z. S. B. & Tasnim, T. An Ensemble hard voting model for cardiovascular disease prediction. in 2nd International Conference on Sustainable Technologies for Industry 4.0 (STI) (2020).
38. Mohapatra, S. et al. A stacking classifiers model for detecting heart irregularities and predicting cardiovascular disease. Healthc. Analytics. 3, 100133 (2023).
39. Saboor, A. et al. A method for improving prediction of human heart disease using machine learning algorithms, Mobile Inf. Syst. 2022, 1410169 (2022).
40. Aldossary, Y., Ebrahim, M. & Hewahi, N. A comparative study of heart disease prediction using tree-based ensemble classification techniques. in International Conference on Data Analytics for Business and Industry (ICDABI) (2022).
41. Duraisamy, P., Natarajan, Y., Ebin, N. L. & Jawahar Raja, P. Efficient way of heart disease prediction and analysis using different ensemble algorithm: A comparative study. in 6th International Conference on Electronics, Communication and Aerospace Technology (2023).
42. Sagi, O. & Rokach, L. Ensemble learning: a survey. WIREs Data Min. Knowl. Discov. 8(4), e1249 (2018).
43. Zhang, C. & Ma, Y. (eds) Ensemble Machine Learning: Methods and Applications (Springer, 2012).
44. Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, 2009).
45. Schapire, R. E. & Singer, Y. Improved boosting algorithms using Confidence-rated predictions. Mach. Learn. 37, 297-336 (1999).
46. Freund, Y. & Schapire, R. E. A Decision-Theoretic generalization of On-Line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119-139 (1997).
47. Weisberg, S. Applied Linear Regression (Wiley, 2005).
48. Kutner, M. H., Nachtsheim, C. J., Neter, J. & Li, W. Applied Linear Regression Models 5th edn (McGraw-Hill Irwin, 2005).
49. Chan, J. Y. L. et al. Mitigating the multicollinearity problem and its machine learning approach: A review. Mathematics (8), 1283 (2022).
50. Hotelling, H. Relations between two sets of variates. Biometrika 28(3/4), 321-377 (1936).
51. Hardoon, D. R., Szedmak, S. & Shawe-Taylor, J. Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16(12), 2639-2664 (2004).
52. James, G., Witten, D., Hastie, T. & Tibshirani, R. An Introduction To Statistical Learning with Applications in R (Springer, 2021).
53. Lundberg, S. M. & Lee, S. I. A unified approach to interpreting model predictions. in 31st International Conference on Neural Information Processing Systems (NIPS’17) (2017).
54. Mane, D. et al. Unlocking machine learning model decisions: A comparative analysis of LIME and SHAP for enhanced interpretability. J. Electr. Syst. 20(2) (2024).
55. Wolpert, D. H. Stacked generalization. Neural Netw. 5(2), 241-259 (1992).
56. Subramani, S. et al. Cardiovascular diseases prediction by machine learning incorporation with deep learning. Front. Med. 10 (2023).
57. Ting, K. M. & Witten, I. H. Issues in stacked generalization. J. Artif. Intell. Res. 10, 271-289 (1999).
58. Abbas, M. A. et al. A novel meta learning based stacked approach for diagnosis of thyroid syndrome. PLOS One (2024).
59. Zhou, Z. H. Ensemble Methods – Foundations and Algorithms (Chapman & Hall/CRC, 2012).
60. Ganie, S. M., Pramanik, P. K. D. & Zhao, Z. Improved liver disease prediction from clinical data through an evaluation of ensemble learning approaches. BMC Med. Inf. Decis. Mak., 24, 160 (2024).
61. Liu, P., Li, L., Li, Y. & Stacking-Based, A. Heart disease classification prediction model. in 10th International Conference on Dependable Systems and Their Applications (DSA) (2023).
62. Zaini, N. A. M. & Awang, M. K. Performance comparison between Meta-classifier algorithms for heart disease classification. Int. J. Adv. Comput. Sci. Appl. 13(10), 323-328 (2022).
63. Chen, Y. W. & Lin, C. J. Combining SVMs with various feature selection strategies, in Feature Extraction. Studies in Fuzziness and Soft Computing, vol. 207, (eds Guyon, I., Nikravesh, M., Gunn, S. & Zadeh, L. A.) Berlin, Heidelberg, Springer, 315-324. (2006).
64. Polat, K. & Güneş, S. A new feature selection method on classification of medical datasets: kernel F-score feature selection. Expert Syst. Appl. 36(7), 10367-10373 (2009).
65. Hodges, J. L. Jr. & Lehmann, E. L. Rank methods for combination of independent experiments in analysis of variance, in Selected Works of E. L. Lehmann. Selected Works in Probability and Statistics, (ed Rojo, J.) Boston, MA, Springer, 403-418. (2012).
66. Holm, S. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6(2), 65-70 (1979).
67. García, S. & Herrera, F. AnExtension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons. J. Mach. Learn. Res. 9, 2677-2694 (2008).
68. Demšar, J. Statistical comparisons of classifiers over multiple datasets. J. Mach. Learn. Res. 7, 1-30 (2006).
69. Band, S. S. et al. Application of explainable artificial intelligence in medical health: A systematic review of interpretability methods. Inf. Med. Unlocked. 40, 101286 (2023).
70. Ponce-Bobadilla, A. V., Schmitt, V., Maier, C. S., Mensing, S. & Stodtmann, S. Practical guide to SHAP analysis: explaining supervised machine learning model predictions in drug development. Clin. Transl. Sci. 17(11), e70056 (2024).

مساهمات المؤلفين

تمويل

الإعلانات

المصالح المتنافسة

معلومات إضافية

OPEN

Ensemble learning with explainable AI for improved heart disease prediction based on multiple datasets

• Designing stacking and voting models for heart disease prediction: We present a comprehensive framework that combines diverse machine learning algorithms to enhance predictive performance.
• Addressing the selection of diverse base models: We consider a range of machine learning algorithms that allow our models to capture the strengths of different approaches and mitigate their weaknesses.
• Conducting experiments on multiple datasets: Our rigorous experimentation was conducted on two distinct heart disease datasets.
• Comprehensive assessment of the models: The proposed ensemble models are rigorously evaluated using various metrics, showcasing their superiority over individual base models and state-of-the-art models.
• Application of rigorous statistical analysis: To ensure the statistical significance of performance improvements, a robust statistical framework-including the Friedman Aligned Ranks test and Holm post-hoc anal-ysis-is implemented.
• Incorporation of Explainable AI (XAI) through SHAP: Our study integrates XAI techniques, specifically SHAP (Shapley Additive Explanations), to interpret the predictions made by the stacking and voting models. This enables us to provide transparency in model predictions and better understand how various features influence the final decision, thereby addressing the interpretability challenges often associated with complex ensemble models.

Related work

• We extensively explored various base models with differing characteristics for developing stacking and voting frameworks.
• We designed unique pipelines to enhance the efficacy, generalisability, and robustness of these stacking and voting frameworks.
• We examined the role of stacking and voting in providing valuable insights into the underlying features and models that influence the final prediction, thereby fostering a better understanding of the disease.
• We employed the Friedman Aligned Ranks test along with Holm’s post-hoc analysis to confirm the statistical significance of the designed model’s performance.
• Unlike many previous studies that regard ensemble models as “black boxes,” our work integrates XAI to provide transparency in model predictions. This sets our study apart by offering interpretable insights into how individual features impact the final prediction, addressing the frequently overlooked issue of model explainability in heart disease prediction.

Research methodology

Research workflow

Stacking and voting

Fig. 1. Proposed methodology for the research.

Fig. 2. Advantages of stacking.

Fig. 3. Advantages of voting.

Stacking

Voting

Constituent models

Weak models

Ensemble models

Dataset for experiment

Fig. 4. Distribution of target variables in both datasets.

Experiment and results

Evaluation metrics

• A true positive (TP) signifies that the patient has heart disease, and the model correctly predicts this.
• A true negative (TN) means that the patient does not have heart disease, and the model accurately predicts it.
• False positives (FPs) indicate that the patient does not have heart disease, yet the model incorrectly predicts a positive result for heart disease.
• A false negative (FN) represents a situation where the patient has heart disease, but the model incorrectly predicts a negative result.

Prediction results of the base models

Pipeline design for stacking and voting

Fig. 7. Performance of the base models with D1.

Fig. 8. Performance of the base models with D2.

Fig. 9. Model selection.

Fig. 10. Pipeline building for stacking and voting.

Fig. 11. Comparative performance of LR, KNN, and LDA as meta-learners.

Input: a) Training/validation dataset $boldsymbol{T}_{boldsymbol{R}}=left{boldsymbol{x}_{boldsymbol{i}}, boldsymbol{y}_{boldsymbol{i}}right}_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}}, boldsymbol{n}$ is no. of instances
    b) Base models: $boldsymbol{B}_{boldsymbol{L}}=left{boldsymbol{b}_{boldsymbol{1}}+boldsymbol{b}_{boldsymbol{2}}+ldots+boldsymbol{b}_{boldsymbol{k}}right}$
    c) A meta-classifier (LR)
Output: Ensemble stacking classifier S
    Initialize an empty 2D array $boldsymbol{S}$ of size $n times k$ to store base learner predictions
    For each base learner $boldsymbol{b}_{boldsymbol{j}}$ in $boldsymbol{B}_{boldsymbol{L}}$ do
        Initialize hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Initialize an empty 1D array $boldsymbol{P}$ to store validation metrics $boldsymbol{P}_{i}$
        For each instance $boldsymbol{i} in{1,2,3, ldots, n}$
            Train $boldsymbol{b}_{boldsymbol{j}}$ on $boldsymbol{T}_{boldsymbol{R}}^{(-boldsymbol{i})}$ // Train all instances except $i^{text {th }}$; keep one for cross-validation
            Predict $boldsymbol{x}_{boldsymbol{i}}$ and compute performance metric $boldsymbol{P}_{boldsymbol{i}}$
            Append $boldsymbol{P}_{boldsymbol{i}}$ to $boldsymbol{P}$
        End
        Compute average validation score $boldsymbol{P}=frac{mathbf{1}}{boldsymbol{n}} sum_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}} boldsymbol{P}_{boldsymbol{i}} / /$ Aggregate performance
        Adjust hyperparameters to maximize $boldsymbol{P}$ // Optimize hyperparameters
        Go to Step 4 until $boldsymbol{P}$ improvement $<1 %$ || Saturation
        Store the best hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Train $boldsymbol{b}_{boldsymbol{j}}$ on full $boldsymbol{T}_{boldsymbol{R}} / /$ Final training
        Perform cross-validation using the remaining instance
        Store the predicted class labels for $boldsymbol{T}_{boldsymbol{R}}$ in column $boldsymbol{j}$ of $boldsymbol{S}$
    End
    Initialize another $2 D$ array $boldsymbol{T}_{boldsymbol{N}}$ of size $n times(k+1) / /$ Prepare meta-dataset
    Concatenate $boldsymbol{S}$ with original labels $boldsymbol{y}: boldsymbol{T}_{boldsymbol{N}}=[boldsymbol{S} boldsymbol{y}]$
    For each instance $boldsymbol{i} in{1,2,3, ldots, n} / /$ Train meta-learner (LR)
        Train $boldsymbol{L} boldsymbol{R}$ on $boldsymbol{T}_{boldsymbol{N}}^{(-boldsymbol{i})}$ // Train all instances except $i^{text {th }}$; keep one for cross-validation
        Predict $boldsymbol{x}_{boldsymbol{i}}$ and store the result in $boldsymbol{S}$
    End
    Return the predicted class labels in $boldsymbol{S}$

Input: a) Training/validation dataset $boldsymbol{T}_{boldsymbol{R}}=left{boldsymbol{x}_{boldsymbol{i}}, boldsymbol{y}_{boldsymbol{i}}right}_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}}, boldsymbol{n}$ is no. of instances
        b) Base models: $boldsymbol{B}_{L}=left{boldsymbol{b}_{1}+boldsymbol{b}_{2}+ldots+boldsymbol{b}_{k}right}$
Output: Ensemble voting classifier $boldsymbol{V}$
    Initialize an empty 2D array $boldsymbol{V}$ of size $n times k$ to store base learner predictions
    Initialize an empty 2D array $boldsymbol{C}$ to store confidence scores for each prediction
    For each base learner $boldsymbol{b}_{boldsymbol{j}}$ in $boldsymbol{B}_{boldsymbol{L}}$ do
        Initialize hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Initialize an empty 1D array $boldsymbol{P}$ to store validation metrics $boldsymbol{P}_{boldsymbol{i}}$
        For each instance $boldsymbol{i} in{1,2,3, ldots, n}$
            Train $boldsymbol{b}_{boldsymbol{j}}$ on $boldsymbol{T}_{boldsymbol{R}}^{(-boldsymbol{i})}$ // Train all instances except $i^{text {th }}$; keep one for cross-validation
            Predict $boldsymbol{x}_{boldsymbol{i}}$ and compute performance metric $boldsymbol{P}_{boldsymbol{i}}$
            Append $boldsymbol{P}_{boldsymbol{i}}$ to $boldsymbol{P}$
        End
        Compute the average validation score $boldsymbol{P}=frac{mathbf{1}}{boldsymbol{n}} sum_{boldsymbol{i}=mathbf{1}}^{boldsymbol{n}} boldsymbol{P}_{boldsymbol{i}} / /$ Aggregate performance
        Adjust hyperparameters to maximize $boldsymbol{P}$ // Optimize hyperparameters
        Go to Step 4 until $boldsymbol{P}$ improvement < $1 %$ // Saturation
        Store the best hyperparameters for $boldsymbol{b}_{boldsymbol{j}}$
        Train $boldsymbol{b}_{boldsymbol{j}}$ on full $boldsymbol{T}_{boldsymbol{R}} / /$ Final training
        Perform cross-validation using the remaining instance
    Store the predicted class labels for $boldsymbol{T}_{R}$ in column $boldsymbol{j}$ of $boldsymbol{V}$ and confidence scores in $boldsymbol{C}$
    End
    Calculate the weighted vote for each class label based on the confidence scores in $boldsymbol{C}$
    Assign the class label with the highest weighted vote to $V$ for this instance
    Return the predicted class labels in $V$

Cross-validation

Assessing feature importance

Fig. 12. The -fold cross-validation process.

Fig. 13. Feature importance of (a) stacking with D1, (b) voting with D1, (c) stacking with D2 and (d) voting with D2.