Linear combination of features reverses importance of all features

I am trying with a logistic model with 2 features independently or with linear combination, but in the linear combination, combining these features would reverse importance through significance levels and regression coefficients and reduce prediction accuracy (AUC).

For example,

With a logistic regression where each of the predictor was scaled with MinMaxScaler

When using each features independently:

Feature A: coefficient = 0.5, P=0.005, AUC=0.59
Feature B: coefficient = 0.1, P=0.5, AUC=0.502

When using linear combination, the statistics are:

Feature A: coefficient: 0.5, P=0.6
Feature B: coefficient: 1.9, P=0.03
== AUC = 0.52

This shows that in the linear combination approach, the less important feature in the single-feature model (Feature B) increase its importance and vice versa for the more important one in the single-feature model (Feature A).

I am not sure how to explain this phenomina,

I would really appreciate if anyone has explanation and comments for this situation.

Thank you very much!

Topic auc logistic-regression

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.