This study explores the effectiveness of binary classification techniques for pneumonia detection from chest X-ray images, comparing traditional machine learning methods with deep learning approaches. A key focus is evaluating cross-dataset generalization, assessing whether models trained on one dataset can accurately classify pneumonia in unseen datasets, a crucial factor for real-world clinical applications.
Using three publicly available datasets—Chest X-Ray Images (Pneumonia), CheXpert, and the RSNA Pneumonia Detection Challenge—the research examines the impact of data variability on model performance. Traditional classifiers, including Logistic Regression, K-Nearest Neighbors, Decision Trees, and Support Vector Machines, are tested alongside deep learning models, such as CNNs and ResNet50. Training and testing on different datasets allow for a comprehensive evaluation of model robustness in handling diverse clinical data.
The findings indicate that deep learning models, particularly ResNet50, outperform traditional methods in both intra-dataset and cross-dataset evaluations. However, simpler models like KNN and Logistic Regression exhibit greater stability across datasets, suggesting that their feature extraction methods are less sensitive to domain shifts. Dataset characteristics, such as class imbalance, label inconsistencies, and image quality, play a significant role in determining performance. Fine-tuning classification thresholds using ROC analysis further enhances diagnostic accuracy.
This study underscores the importance of AI-driven diagnostic tools while highlighting the challenges of ensuring model generalization. Future research will focus on integrating multimodal data, addressing dataset biases, and improving model interpretability to enhance AI-assisted pneumonia detection in clinical practice.