Feature importance difference in two similar machine learning models
Situation 1:
I have trained a text classification model (Model 1) which gives me a probability of true class as X. I have also trained a classification model (Model 2) using only the categorical and numeric data. Both the models are used to predict the same true class; just the features differ. I used a random forest classifier on the probabilities returned by Model 1 and Model 2(taking them as input features) and got similar performance metrics(Accuracy, Precision recall). feature importance was 49% for model 1 and 51% for model 2.
Situation 2:
I used the probability X of text classification model as an input feature to the Model 2(which contained categorical and numeric features). The performance was almost similar to Situation 1 but here the feature importance of the final model indicated that text model probability had higher importance around 68% and rest of the features had lesser importance.
I want to understand the difference in feature importance of both situations.
Topic features stacking xgboost random-forest machine-learning
Category Data Science