If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree

Question

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree

user6703592

2021年11月7日 18:16

I have asked this question here, but seems no one was interested in it:

https://stats.stackexchange.com/questions/550994/if-a-feature-has-already-split-will-it-hardly-be-selected-to-split-again-in-the

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree? It is motivated by the fact that for the heavy correlated features in a single tree, usually only one of them will be selected to split as their uncertainty will remain few after a splitting. Now in Gradient Boosting Tree, is residual similar with the uncertainty?

Currently, I happened to how heavily correlated features affects the feature importance selected by Gradient Boosting Tree. I guess the result is that Gradient Boosting Tree will only select the importance from one of correlated features just like LASSO.

Topic gradient-boosting-decision-trees boosting decision-trees random-forest machine-learning

Category Data Science

Carlos Mougan · Accepted Answer · 2021年11月7日 18:16

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree?

It's harder yes, but common. In the same tree it can not happen in the same point. In a subsequent tree it can.

It is motivated by the fact that for the heavy correlated features in a single tree, usually only one of them will be selected to split as their uncertainty will remain few after a splitting. Now in Gradient Boosting Tree, is residual similar with the uncertainty?

If you are trying to understand gradient boosting I would not mix it with uncertainty. There are several ways to compute the uncertainty with gradient boosting, NGBoost is one but not the only.

Your question needs further retunnign to properly be answered. You are mixing highly correlated features, with uncertainty. You need to clarify them before attempting to answer this question.

If a feature has already split, will it hardly be selected to split again in the subsequent tree in a Gradient Boosting Tree

About