Do monotonic constraints prevent an XGboost to capture non-linear relationships in the data?
I have trained an XGBoost model (for a binary classification problem) and I have tested two scenarios:
Scenario 1 - No Monotonic Constrained applied
In this case I get a Gini on the training sample of 81.1 and a Gini on the Validation sample of 76.5, throwing a red flag for overfitting.
I have taken a look at the SHAP dependence plot for one char in Scenario 1 and it looks like this:
Scenario 2 - Monotonic Constrained applied
In this case I get a Gini on the training sample of 69.0 and a Gini on the Validation sample of 68.7, which looks like the model has been generalized well.
I have taken a look at the SHAP dependence plot for the same char as above in Scenario 2 and it looks like this:
I was wondering if the application of monotonic constraints to the predictive features actually prevents the XGBoost Classifier to capture the non-linear relationship between the predictive features and the binary target? Intuitively I'd say yes, but when I look at the SHAP plots I don't think so anymore. Can anyone clarify this for me please?
Topic shap overfitting xgboost
Category Data Science