2

I built a predictive model using an elastic net regression model with sklearn. The model R2 = 0.015. I know SHAP method could provide the importance of the features. However, How to calculate the significance of each feature? (Get which feature is significant or which features successfully predict the response.This way, I can tell my story in the paper and discuss these features in detail.)

As far as I know, R package "eNetXplorer" can do this by permutation test, but I have identified a useful elastic net model via Scikit-learn.Is there a similar package in the python environment?

Any help is greatly appreciated!

Kengo Ito
  • 23
  • 4

2 Answers2

2

As far as I have been able to discover, there are NO statistical summaries in any of the SKlearn Regression modeling packages. You can get r-squared values and some metrics and create some other metrics yourself. That is a limit of SKlearn regressions.

That having been said, ElasticNet does not really support a traditional summary. The regularization in the model chooses the most predictive variables and penalizes to reduce variance in the models and converge on 'the best' coefficients. You can see how it evolves as it iterates through models to converge on RMSE, but other than that, there is nothing.

The model is not using ordinary least squares residuals to optimize the coefficients, so all of the outputs you are used to seeing in a traditional OLS are not available to you. That is something you give up by using an Elastic Net (as well as bias introduced by the regularization).

sconfluentus
  • 606
  • 4
  • 6
1

Assuming you are talking about getting a features impact scores ranking (i.e. sort features by their relevance on model predictions), I would go for a permutation importance methodology. It is a model-agnostic approach which you can use providing an already fit model and a evaluation dataset.

The concept is to relate the highest drops in model performance with the most important features, when the values of these ones are shuffled. The process it follows could be defined by the following steps:

  1. Make model predictions on a sample of records
  2. Select a column, shuffle its values and predict again with the model
  3. Get the the drop (if any) of model performance on this new shuffled dataset VS the initial one
  4. Average this difference across all predictions
  5. Repeat the steps above for all features

Scikit-learn provides something like this with its permutation importance functionality, I hope it helps. Other source of info here

German C M
  • 2,744
  • 7
  • 18