In stacked generalization, several algorithms (I use some random trees, booster trees, etc.) are first trained and used to make the predictions which are used as input for another algorithm. However, can I use any kind of algorithms, or is there a preference?
P.S.: I often see people using linear models in this case.
Topic stacking ensemble-modeling algorithms
Category Data Science
There is no preference for stacked generalization. You can use any algorithm whether it be a NN, RF model, XGB model, etc. The only thing that you need to take care of is the fact that the algorithm which you are applying to your model is useful to your data or not. Also, model stacked at level 1 may not be a very good model for level 2. For example, suppose at level 1 you stacked an XGB model with max_depth=12, eta=0.01, colsample_bytree=0.7, but at second level it might happen that the same depth is just an overkill and hence you should discard using that model at second level. So, in short there is no preference for stacked models, the preference is for the data on which you are going to train your model.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.