# predictorImportance

Estimates of predictor importance

## Syntax

```imp = predictorImportance(ens)[imp,ma] = predictorImportance(ens)```

## Description

`imp = predictorImportance(ens)` computes estimates of predictor importance for `ens` by summing these estimates over all weak learners in the ensemble. `imp` has one element for each input predictor in the data used to train this ensemble. A high value indicates that this predictor is important for `ens`.

```[imp,ma] = predictorImportance(ens)``` returns a `P`-by-`P` matrix with predictive measures of association for `P` predictors.

## Input Arguments

 `ens` A regression ensemble created by `fitensemble`, or by the `compact` method.

## Output Arguments

 `imp` A row vector with the same number of elements as the number of predictors (columns) in `ens``.X`. The entries are the estimates of predictor importance, with `0` representing the smallest possible importance. `ma` A `P`-by-`P` matrix of predictive measures of association for `P` predictors. Element `ma(I,J)` is the predictive measure of association averaged over surrogate splits on predictor `J` for which predictor `I` is the optimal split predictor. `predictorImportance` averages this predictive measure of association over all trees in the ensemble.

## Definitions

### Predictor Importance

`predictorImportance` computes estimates of predictor importance for `tree` by summing changes in the mean squared error (MSE) due to splits on every predictor and dividing the sum by the number of branch nodes. If the tree is grown without surrogate splits, this sum is taken over best splits found at each branch node. If the tree is grown with surrogate splits, this sum is taken over all splits at each branch node including surrogate splits. `imp` has one element for each input predictor in the data used to train this tree. At each node, MSE is estimated as node error weighted by the node probability. Variable importance associated with this split is computed as the difference between MSE for the parent node and the total MSE for the two children.

### Predictive Measure of Association

The predictive measure of association between the optimal split on variable i and a surrogate split on variable j is:

${\lambda }_{i,j}=\frac{\text{min}\left({P}_{L},{P}_{R}\right)-\left(1-{P}_{{L}_{i}{L}_{j}}-{P}_{{R}_{i}{R}_{j}}\right)}{\text{min}\left({P}_{L},{P}_{R}\right)}.$

Here

• PL and PR are the node probabilities for the optimal split of node i into Left and Right nodes respectively.

• ${P}_{{L}_{i}{L}_{j}}$ is the probability that both (optimal) node i and (surrogate) node j send an observation to the Left.

• ${P}_{{R}_{i}{R}_{j}}$ is the probability that both (optimal) node i and (surrogate) node j send an observation to the Right.

Clearly, λi,j lies from –∞ to 1. Variable j is a worthwhile surrogate split for variable i if λi,j > 0.

Element `ma(i,j)` is the predictive measure of association averaged over surrogate splits on predictor `j` for which predictor `i` is the optimal split predictor. This average is computed by summing positive values of the predictive measure of association over optimal splits on predictor `i` and surrogate splits on predictor `j` and dividing by the total number of optimal splits on predictor `i`, including splits for which the predictive measure of association between predictors `i` and `j` is negative.

## Examples

Estimate the predictor importance for all numeric variables in the `carsmall` data:

```load carsmall X = [Acceleration Cylinders Displacement ... Horsepower Model_Year Weight]; ens = fitensemble(X,MPG,'LSBoost',100,'Tree'); imp = predictorImportance(ens) imp = 0.0082 0 0.0049 0.0133 0.0142 0.1737```

The weight (last predictor) has the most impact on mileage (MPG). The second predictor has importance 0; this means the number of cylinders has no impact on predictions made with `ens`.

Estimate the predictor importance for all variables in the `carsmall` data for an ensemble where the trees contain surrogate splits:

```load carsmall surrtree = templateTree('Surrogate','on'); X = [Acceleration Cylinders Displacement ... Horsepower Model_Year Weight]; ens2 = fitensemble(X,MPG,'LSBoost',100,surrtree); [imp2,ma] = predictorImportance(ens2) imp2 = 0.0725 0.1342 0.1425 0.1397 0.1380 0.1855 ma = 1.0000 0.0414 0.0607 0.0782 0.0102 0.0322 0 1.0000 0 0 0 0 0.0441 0.0704 1.0000 0.0883 0.0175 0.0913 0.0944 0.1166 0.1400 1.0000 0.0390 0.1308 0.0121 0.0139 0.0127 0.0127 1.0000 0.0113 0.0818 0.1317 0.2072 0.1878 0.0340 1.0000```

While weight (last predictor) still has the most impact on mileage (MPG), this estimate has the second predictor (number of cylinders) is essentially tied for third most important predictor.