Why does SHAP's TreeExplainer with "interventional" method not match exact SHAP?
I am trying to understand the concepts/definitions behind the SHAP method of explaining model predictions. In particular I've read the original SHAP paper and the TreeExplainer paper. The original paper lays out a particular, well-defined set of values for a given model prediction on tabular data, which can be computed exactly (although this is very slow in practice, so the paper/package gives various other algorithms as approximations.)
In the TreeExplainer paper, algorithm 1 2 for TreeExplainer with path-dependent feature perturbation are explicitly acknowledged to be computing a different definition. However algorithm 3 (interventional Tree SHAP), it claims, exactly computes the traditional Shapley value formula. The paper/package author Scott Lundberg stated again in this GH discussion:
there is no disagreement between
TreeExplainer(model, prior, feature_perturbation=interventional).shap_values(X)
vs.KernelExplainer(model.predict, prior).shap_values(X, nsamples=Inf)
(KernelExplainer is an approximation which is exact when nsamples=Inf).
However, I checked in Python on a toy dataset tree model that TreeExplainer(model, data, feature_perturbation=interventional).shap_values(X)
is giving substantially different values than shap.explainers.Exact(model.predict_proba, data)
! Am I missing something, or does interventional TreeExplainer compute something that is again defined differently than original SHAP?
Topic shap interpretation explainable-ai machine-learning
Category Data Science