ENHANCING DIAGNOSTIC ACCURACY WITH ENSEMBLE TECHNIQUES: DETECTING COVID-19 AND PNEUMONIA ON CHEST X-RAY IMAGES

(Received: 27-May-2024, Revised: 17-Jul.-2024 , Accepted: 12-Aug.-2024)

Authors Fatma A. Mostafa, Lamiaa A. Elrefaei, Mostafa M. Fouda, Aya Hossam,

Keywords #COVID-19 #CXR #DenseNet-169 #ViT-l32 #Stacking #Boosting #Bagging #Blending

Abstract Lung diseases such as COVID-19 and pneumonia can lead to severe complications, including breathing difficulties, decreased lung function and respiratory failure, which can be life-threatening if not promptly treated. Chest X-ray imaging techniques have proven to be quick, effective and cost-efficient in diagnosing and monitoring these diseases. Additionally, artificial intelligence, particularly through deep learning and machine learning, has shown promising results in detecting various lung diseases, including COVID-19 and pneumonia. This technology’s ability to analyze large datasets rapidly has contributed to reducing the spread of these diseases and has significantly advanced biomedical research in various medical disciplines. In this research paper, we introduced various advanced ensemble techniques as bagging, boosting, stacking and blending with different algorithms, to enhance the performance of our classification models in detecting coronavirus and pneumonia. We specifically focused on combining convolutional neural network (CNN) and vision transformer (ViT) models to create powerful ensemble models. Our objective was to determine the most accurate ensemble technique for diagnosing lung diseases. We assessed their ability to correctly classify chest X-ray images as either COVID-19, pneumonia or normal. The CatBoost model achieved the highest accuracy, F1-score and ROC-AUC score of 99.753%, 99.51% and 99.99%, respectively using the COVID-19 Radiography dataset. The bagging ensemble model achieved the highest accuracy, F1-score and ROC-AUC score of 95.08%, 95.2% and 99.69%, respectively using COVIDx CXR-4. The results indicate that the advanced ensemble techniques can significantly improve the performance of machine-learning models.

References

[1] F. A. Mostafa, L. A. Elrefaei, M. M. Fouda and A. Hossam, "A Survey on AI Techniques for Thoracic Diseases Diagnosis Using Medical Images," Diagnostics, vol. 12, no. 12, p. 3034, 2022.

[2] I. Team, "Coronavirus Cases," [Online]. Available: https://www.worldometers.info/ coronavirus/, 2024.

[3] F. A. Mostafa, L. A. Elrefaei, M. M. Fouda and A. Hossam, "Diagnosis of Lung Diseases from Chest X- Ray Images Using Different Fusion Techniques," Proc. of the 2023 11th Int. Conf. on Information and Communication Technology (ICoICT), pp. 429–435, Melaka, Malaysia, 2023.

[4] M. H. Saad, S. Hashima, W. Sayed, E. H. El-Shazly, A. H. Madian and M. M. Fouda, "Early Diagnosis of COVID-19 Images Using Optimal CNN Hyper-parameters," Diagnostics, vol. 13, no. 1, p. 76, 2023.

[5] Z. Ahmad, S. Rahim, M. Zubair and J. Abdul-Ghafar, "Artificial Intelligence (AI) in Medicine, Current Applications and Future Role with Special Emphasis on Its Potential and Promise in Pathology: Present and Future Impact, Obstacles Including Costs and Acceptance among Pathologists, Practical and Philosophical Considerations: A Comprehensive Review," Diagnostic pathology, vol. 16, pp. 1–16, 2021.

[6] H. Q. Nguyen et al., "VinDr-CXR: An Open Dataset of Chest X-Rays with Radiologist’s Annotations," Scientific Data, vol. 9, no. 1, p. 429, 2022.

[7] A. Shrestha and A. Mahmood, "Review of Deep Learning Algorithms and Architectures," IEEE Access, vol. 7, pp. 53040–53065, 2019.

[8] Z. Yu, K. Wang, Z. Wan, S. Xie and Z. Lv, "Popular Deep Learning Algorithms for Disease Prediction: A Review," Cluster Computing, vol. 26, no. 2, pp. 1231–1251, 2023.

[9] A. A. Chowdhury, K. T. Hasan and K. K. S. Hoque, "Analysis and Prediction of COVID-19 Pandemic in Bangladesh by Using ANFIS and LSTM Network," Cognitive Computation, vol. 13, pp. 761-770, [Online], Available: https://link.springer.com/article/10.1007/s12559-021-09859-0, Apr. 2021.

[10] D. Kuzinkovas and S. Clement, "The Detection of COVID-19 in Chest X-Rays Using Ensemble CNN Techniques," Information, vol. 14, no. 7, p. 370, 2023.

[11] A. Zizaan and A. Idri, "Evaluating and Comparing Bagging and Boosting of Hybrid Learning for Breast Cancer Screening," Scientific African, vol. 23, p. e01989, DOI: 10.1016/j.sciaf.2023.e01989, 2024.

[12] U. Ahmed, J. C.-W. Lin and G. Srivastava, "Towards Early Diagnosis and Intervention: An Ensemble Voting Model for Precise Vital Sign Prediction in Respiratory Disease," IEEE Journal of Biomedical and Health Informatics, pp. 1-13, DOI: 10.1109/JBHI.2023.3270888, 2023.

[13] U. Bhimavarapu, N. Chintalapudi and G. Battineni, "Multi-classification of Lung Infections Using Improved Stacking Convolution Neural Network," Technologies, vol. 11, no. 5, p. 128, 2023.

[14] T. Mahesh et al., "Blended Ensemble Learning Prediction Model for Strengthening Diagnosis and Treatment of Chronic Diabetes Disease," Computational Intelligence and Neuroscience, vol. 2022, DOI: 10.1155/2022/4451792, 2022.

[15] T. Mahesh et al., "Early Predictive Model for Breast Cancer Classification Using Blended Ensemble Learning," Int. J. of System Assurance Eng. and Management, vol. 15, no. 1, pp. 188–197, 2024.

[16] R. Hasan, S. M. Azmat Ullah, A. Nandi and A. Taher, "Improving Pneumonia Diagnosis: A Deep Transfer Learning CNN Ensemble Approach for Accurate Chest X-ray Image Analysis," Proc. of the 2023 Int. Conf. on Information and Communication Technology for Sustainable Development (ICICT4SD), pp. 109–113, hal-04163800f, 2023.

[17] S. Tang, C. Wang, J. Nie, N. Kumar, Y. Zhang, Z. Xiong and A. Barnawi, "EDL-COVID: Ensemble Deep Learning for COVID-19 Case Detection from Chest X-ray Images," IEEE Transactions on Industrial Informatics, vol. 17, no. 9, pp. 6539–6549, 2021.

[18] S. Govindarajan, S. R. Manuskandan and R. Swaminathan, "Diagnostics of Multi Drug Resistant Tuberculosis in Chest Radiographs Using Local Textures & Extreme Gradient Boosting," Current Directions in Biomedical Engineering, vol. 9, no. 1, pp. 721–724, 2023.

[19] S. Kalaivani and K. Seetharaman, "A Three-stage Ensemble Boosted Convolutional Neural Network for Classification and Analysis of COVID-19 Chest X-ray Images," Int. Journal of Cognitive Computing in Engineering, vol. 3, pp. 35–45, 2022.

[20] R. Soundrapandiyan, H. Naidu, M. Karuppiah, M. Maheswari and R. C. Poonia, "AI-based Wavelet and Stacked Deep Learning Architecture for Detecting Coronavirus (COVID-19) from Chest X-ray Images," Computers and Electrical Engineering, vol. 108, Article no. 108711, 2023.

[21] P. Mooney, "Chest X-ray Images (Pneumonia)," Kaggle, [Online], Available: https://www.kaggle.com/datasets/paultimothymooney/chest-xray-pneumonia, Mar. 2018.

[22] M.-L. Huang and Y.-C. Liao, "Stacking Ensemble and ECA-EfficientNetV2 Convolutional Neural Networks on Classification of Multiple Chest Diseases Including COVID-19," Academic Radiology, vol. 30, no. 9, pp. 1915–1935, 2023.

[23] G. Erol and B. Uzbaş, "Detection of COVID-19 Severity and Mortality from Blood Parameters by Ensemble Learning Methods," Karaelmas Fen ve Mühendislik Dergisi, vol. 13, p. 316–328, 2023.

[24] A. Banerjee, A. Sarkar, S. Roy, P. K. Singh and R. Sarkar, "Covid-19 Chest X-ray Detection through Blending Ensemble of CNN Snapshots," Biomedical Signal Processing and Control, vol. 78, p. 104000, 2022.

[25] Peshotan, "Summary of COVID-19 Radiography Database," [Online], Available: Https://github.com/sfu-db/covid19-datasets/blob/master/datasets-details/radiography-kaggle-COVID-19-dataset.md, 2020.

[26] T. Pham, "Classification of COVID-19 Chest X-rays with Deep Learning: New Models or Fine Tuning?" Health Information Science and Systems, vol. 9, no. 1, DOI: 10.1007/s13755-020-00135-3, 2020.

[27] T. Rahman, "Covid-19 Radiography Database," [Online], Available: https: /www.kaggle.com/datasets/tawsifurrahman/covid19-radiography-database, Mar. 2022.

[28] Y. Wu, H. Gunraj, C. en Amy Tai and A. Wong, "COVIDx CXR-4: An Expanded Multi-institutional Open-source Benchmark Dataset for Chest X-ray Image-based Computer-aided COVID-19 Diagnostics," arViv: 2311.17677, DOI: arxiv-2311.17677, 2023.

[29] P. P. Dalvi, D. R. Edla and B. R. Purushothama, "Diagnosis of Coronavirus Disease from Chest X-ray Images Using Densenet-169 Architecture," SN Computer Science, vol. 4, Article no. 214, [Online], Available: https://link.springer.com/article/10.1007/s42979-022-01627-7, Feb. 2023.

[30] M. K. Bohmrah and H. Kaur, "Classification of COVID-19 Patients Using Efficient Fine-tuned Deep Learning DenseNet Model," Global Transitions Proceedings, vol. 2, no. 2, pp. 476–483, 2021.

[31] A. Ala and Polat, "Detection of COVID-19 from Computed Tomography Images with DenseNet based Deep Learning Models," Proc. of the 2021 29th Signal Processing and Communications Applications Conference (SIU), pp. 1-4, Istanbul, Turkey, 2021.

[32] K. Han et al., "A Survey on Vision Transformer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 87–110, 2023.

[33] S. Park et al., "Multi-task Vision Transformer Using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantification," Medical Image Analysis, vol. 75, Article no. 102299, 2022.

[34] C. C. Ukwuoma et al., "Automated Lung-related Pneumonia and COVID-19 Detection Based on Novel Feature Extraction Framework and Vision Transformer Approaches Using Chest X-ray Images," Bioengineering, vol. 9, no. 11, Article no. 709, 2022.

[35] T. Aitazaz, A. Tubaishat, F. Al-Obeidat, B. Shah, T. Zia and A. Tariq, "Transfer Learning for Histopathology Images: An Empirical Study," Neural Computing and Applications, vol. 35, pp. 7963-7974, Jul. 2022.

[36] G. Boesch, "Vision Transformers (ViT) in Image Recognition," 2024 Guide, [Online], Available: https://viso.ai/deep-learning/vision-transformer-vit/, Jun. 2024.

[37] R. Kundu, R. Das, Z. W. Geem, G.-T. Han and R. Sarkar, “Pneumonia Detection in Chest X-ray Images Using an Ensemble of Deep Learning Models,” PloS One, vol. 16, no. 9, p. e0256630, 2021.

[38] S. Kaleem, A. Sohail, M. U. Tariq, M. Babar and B. Qureshi, “Ensemble Learning for Multi-class COVID-19 Detection from Big Data,” Plos One, vol. 18, no. 10, p. e0292587, 2023.

[39] O. O. Abayomi-Alli et al., “An Ensemble Learning Model for COVID-19 Detection from Blood Test Samples,” Sensors, vol. 22, no. 6, Article no. 2224, 2022.

[40] S. Shukla, K. Arya, B. Garg and P. Lezama, “Pneumonia Detection Using Gradient Boosting on Selected Features Extracted from DenseNet,” Proc. of the 7th ASRES Int. Conf. on Intelligent Technologies, in Book Series: Lecture Notes in Networks and Systems, vol. 685, pp. 181–195, 2022.

[41] A. S. Ashour, M. M. Eissa, M. A. Wahba, R. A. Elsawy, H. F. Elgnainy, M. S. Tolba and W. S. Mohamed, “Ensemble-based Bag of Features for Automated Classification of Normal and COVID-19 CXR Images,” Biomedical Signal Processing and Control, vol. 68, p. 102656, 2021.

[42] M. Galar, A. Fernandez, E. Barrenechea, H. Bustince and F. Herrera, “A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting- and Hybrid-based Approaches,” IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 4, pp. 463–484, 2011.

[43] R. Odegua, “An Empirical Study of Ensemble Techniques (Bagging, Boosting and Stacking),” Proc. of Deep Learning Conf. (IndabaXAt), DOI: 10.13140/RG.2.2.35180.10882, 2019.

[44] Z. Hu, H. Qiu, Z. Su, M. Shen and Z. Chen, “A Stacking Ensemble Model to Predict Daily Number of Hospital Admissions for Cardiovascular Diseases,” IEEE Access, vol. 8, pp. 138719–138729, 2020.

[45] M. Gour and S. Jain, “Automated COVID-19 Detection from X-ray and CT Images with Stacked Ensemble Convolutional Neural Network,” Biocybernetics and Biomedical Engineering, vol. 42, no. 1, pp. 27–41, 2022.

[46] A. A. Hammam, H. H. Elmousalami and A. E. Hassanien, “Stacking Deep Learning for Early COVID- 19 Vision Diagnosis,” in Chapter: Big Data Analytics and Artificial Intelligence Against COVID-19: Innovation Vision and Approach, vol. 78, pp. 297–307, DOI: 10.1007/978-3-030-55258-9_18, 2020.

[47] J. A. ALzubi, B. Bharathikannan, S. Tanwar, R. Manikandan, A. Khanna and C. Thaventhiran, “Boosted Neural Network Ensemble Classification for Lung Cancer Disease Diagnosis,” Applied Soft Computing, vol. 80, pp. 579–591, 2019.

[48] G. Li, R. Togo, T. Ogawa and M. Haseyama, “Boosting Automatic COVID-19 Detection Performance with Self-supervised Learning and Batch Knowledge Ensembling,” Computers in Biology and Medicine, vol. 158, Article no. 106877, 2023.

[49] N. Habib, M. M. Hasan, M. M. Reza and M. M. Rahman, “Ensemble of CheXNet and VGG-19 Feature Extractor with Random Forest Classifier for Pediatric Pneumonia Detection,” SN Computer Science, vol. 1, no. 6, Article no. 359, 2020.

[50] A. B. Godbin and S. G. Jasmine, “A Machine Learning Based Approach for Diagnosing Pneumonia with Boosting Techniques,” in Chapter: Machine Intelligence for Smart Applications, Part of the Book Series: Studies in Computational Intelligence, vol. 1105, pp. 145–160, Springer, 2023.