Documentation

This is machine translation

Translated by Microsoft
Mouse over text to see original. Click the button below to return to the English verison of the page.

predictorImportance

Class: CompactRegressionEnsemble

Estimates of predictor importance

Syntax

imp = predictorImportance(ens)
[imp,ma] = predictorImportance(ens)

Description

imp = predictorImportance(ens) computes estimates of predictor importance for ens by summing these estimates over all weak learners in the ensemble. imp has one element for each input predictor in the data used to train this ensemble. A high value indicates that this predictor is important for ens.

[imp,ma] = predictorImportance(ens) returns a P-by-P matrix with predictive measures of association for P predictors.

Input Arguments

ens

A regression ensemble created by fitensemble, or by the compact method.

Output Arguments

imp

A row vector with the same number of elements as the number of predictors (columns) in ens.X. The entries are the estimates of predictor importance, with 0 representing the smallest possible importance.

ma

A P-by-P matrix of predictive measures of association for P predictors. Element ma(I,J) is the predictive measure of association averaged over surrogate splits on predictor J for which predictor I is the optimal split predictor. predictorImportance averages this predictive measure of association over all trees in the ensemble.

Definitions

Predictor Importance

predictorImportance computes estimates of predictor importance for tree by summing changes in the mean squared error (MSE) due to splits on every predictor and dividing the sum by the number of branch nodes. If the tree is grown without surrogate splits, this sum is taken over best splits found at each branch node. If the tree is grown with surrogate splits, this sum is taken over all splits at each branch node including surrogate splits. imp has one element for each input predictor in the data used to train this tree. At each node, MSE is estimated as node error weighted by the node probability. Variable importance associated with this split is computed as the difference between MSE for the parent node and the total MSE for the two children.

Predictive Measure of Association

The predictive measure of association is a value that indicates the similarity between decision rules that split observations. Among all possible decision splits that are compared to the optimal split (found by growing the tree), the best surrogate decision split yields the maximum predictive measure of association. The second-best surrogate split has the second-largest predictive measure of association.

Suppose xj and xk are predictor variables j and k, respectively, and jk. At node t, the predictive measure of association between the optimal split xj < u and a surrogate split xk < v is

λjk=min(PL,PR)(1PLjLkPRjRk)min(PL,PR).

  • PL is the proportion of observations in node t, such that xj < u. The subscript L stands for the left child of node t.

  • PR is the proportion of observations in node t, such that xju. The subscript R stands for the right child of node t.

  • PLjLk is the proportion of observations at node t, such that xj < u and xk < v.

  • PRjRk is the proportion of observations at node t, such that xju and xkv.

  • Observations with missing values for xj or xk do not contribute to the proportion calculations.

λjk is a value in (–∞,1]. If λjk > 0, then xk < v is a worthwhile surrogate split for xj < u.

Algorithms

Element ma(i,j) is the predictive measure of association averaged over surrogate splits on predictor j for which predictor i is the optimal split predictor. This average is computed by summing positive values of the predictive measure of association over optimal splits on predictor i and surrogate splits on predictor j and dividing by the total number of optimal splits on predictor i, including splits for which the predictive measure of association between predictors i and j is negative.

Examples

expand all

Estimate the predictor importance for all predictor variables in the data.

Load the carsmall data set.

load carsmall

Grow an ensemble of 100 regression trees for MPG using Acceleration, Cylinders, Displacement, Horsepower, Model_Year, and Weight as predictors.

X = [Acceleration Cylinders Displacement Horsepower Model_Year Weight];
ens = fitensemble(X,MPG,'LSBoost',100,'Tree');

Estimate the predictor importance for all predictor variables.

imp = predictorImportance(ens)
imp =

    0.0150         0    0.0066    0.1111    0.0437    0.5181

Weight, the last predictor, has the most impact on mileage. The second predictor has importance 0, which means that the number of cylinders has no impact on predictions made with ens.

Estimate the predictor importance for all variables in the data and where the regression tree ensemble contains surrogate splits.

Load the carsmall data set.

load carsmall

Grow an ensemble of 100 regression trees for MPG using Acceleration, Cylinders, Displacement, Horsepower, Model_Year, and Weight as predictors. Specify to identify surrogate splits.

X = [Acceleration Cylinders Displacement Horsepower Model_Year Weight];
t = templateTree('Surrogate','on');
ens = fitensemble(X,MPG,'LSBoost',100,t);

Estimate the predictor importance and predictive measures of importance for all predictor variables.

[imp,ma] = predictorImportance(ens)
imp =

    0.2065    0.3799    0.4100    0.6190    0.3670    0.5476


ma =

    1.0000    0.0098    0.0102    0.0098    0.0033    0.0067
         0    1.0000         0         0         0         0
    0.0056    0.0084    1.0000    0.0078    0.0022    0.0084
    0.2232    0.1574    0.2066    1.0000    0.0580    0.2059
    0.0061    0.0070    0.0063    0.0064    1.0000    0.0056
    0.0200    0.0368    0.0620    0.0521    0.0098    1.0000

Comparing imp to the results in Estimate Predictor Importance, Horsepower has the greatest impact on mileage, with Weight having the second greatest impact.

Was this topic helpful?