Fit Decision Tree to Gradient Boosted Trees for Interpretability

Question

Fit Decision Tree to Gradient Boosted Trees for Interpretability

jtanman

2021年3月23日 13:25

I was wondering if there is literature on or someone could explain how to fit a decision tree to a gradient boosted trees classifier in order to derive more interpretable results.

This is apparently the approach that Turi uses in their explain function which outputs something like this:

Turi's explain function:

from their page here.

I know that for random forests you can average the contribution of each feature in every tree as seen in the TreeInterpreter python package, but this seems to be a different method since it is focused on exact splits and one decision tree.

Does anyone know anything more about this method for interpreting gradient boosted trees?

Topic decision-trees gbm classification python

Category Data Science

Pieter · Accepted Answer · 2016年8月8日 22:54

Gradient boosting learns multiple decision or regression trees after each other. The difference with random forests is that the trees correct each other. Each new tree is fitted on the residual produced by the predictions from the earlier trees.

The explain method shows for each prediction (i.e. record) why a particular decision was made. This results in a set of decisions. Each tree makes several decisions. The decisions of all trees are stacked to get the set of rules that justifies a single decision. This information is specific for the gradient boosting model and the specific record.

If you want to know more about the gradient boosting model, you may want to consider feature importance. Feature importance expresses how important the features are. You can use print model.get_feature_importance() for this.

Fit Decision Tree to Gradient Boosted Trees for Interpretability

About