PCA Impact on Logistic Regression and KNN

Log Reg PCA: When we introduced PCA to the Logistic Regression model, the performance

metrics remained virtually unchanged. The accuracy only nudged up slightly to 80.33%,

and the Recall stayed around 94%. This finding suggests that for Logistic Regression,

the original features were already highly informative and not overly correlated, so reducing the

dimensionality via PCA did not provide significant additional value in this particular dataset.

kNN no PCA: Next, we moved to the K-Nearest Neighbors model, tuned with 11 nearest

neighbors. The performance immediately improved significantly, achieving an overall

Accuracy of 88.52%. More importantly, the Sensitivity, or Recall, climbed to 0.9697.