I'm currently working on a project where the ultimate goal is to reduce the quantity of a bad thing, b.
I've been tasked with assigning the blame between several different features that are believed to increase b. Then, we can appropriately prioritise improving the features that cause the largest increases.
The first method I tried was using SciPy's lsq_linear optimizer. I fixed the intercept at zero, and fixed the coefficients of each feature to be greater than or equal to zero. I then used the model to predict b. I can then say that, for a given set of feature values, exactly how much each feature's value contributed to increasing the prediction of b. Importantly, the sum of these contributions adds up to the prediction itself.
However, my problem is that the relationship between many features and b is not linear. I've found that a tree based model like XGBoost produces a more accurate predictions. However, I'm struggling to find a way to interpret a tree-based model in a similar way to the lsq_linear model.
I've used Shapley values to interpret tree-based models before, but they aren't appropriate here as the sum of the Shapley values do not add up to the prediction itself, but to the difference between each prediction and the average prediction.
Is there a method that I can use to appropriately assign blame to each feature in a way that the sum of the contributions equals the model output of a tree-based model?
