Documentation

This is machine translation

Translated by Microsoft
Mouse over text to see original. Click the button below to return to the English verison of the page.

predictorImportance

Class: CompactClassificationEnsemble

Estimates of predictor importance

Syntax

imp = predictorImportance(ens)
[imp,ma] = predictorImportance(ens)

Description

imp = predictorImportance(ens) computes estimates of predictor importance for ens by summing these estimates over all weak learners in the ensemble. imp has one element for each input predictor in the data used to train this ensemble. A high value indicates that this predictor is important for ens.

[imp,ma] = predictorImportance(ens) returns a P-by-P matrix with predictive measures of association for P predictors, when the learners in ens contain surrogate splits. See Definitions.

Input Arguments

ens

A classification ensemble created by fitensemble, or by the compact method.

Output Arguments

imp

A row vector with the same number of elements as the number of predictors (columns) in ens.X. The entries are the estimates of predictor importance, with 0 representing the smallest possible importance.

ma

A P-by-P matrix of predictive measures of association for P predictors. Element ma(I,J) is the predictive measure of association averaged over surrogate splits on predictor J for which predictor I is the optimal split predictor. predictorImportance averages this predictive measure of association over all trees in the ensemble.

Definitions

Predictor Importance

predictorImportance computes estimates of predictor importance for ens by summing changes in the risk due to splits on every predictor and dividing the sum by the number of branch nodes. If ens is grown without surrogate splits, this sum is taken over best splits found at each branch node. If ens is grown with surrogate splits, this sum is taken over all splits at each branch node including surrogate splits. imp has one element for each input predictor in the data used to train ens. Predictor importance associated with this split is computed as the difference between the risk for the parent node and the total risk for the two children.

Impurity and Node Error

ClassificationTree splits nodes based on either impurity or node error.

Impurity means one of several things, depending on your choice of the SplitCriterion name-value pair argument:

  • Gini's Diversity Index (gdi) — The Gini index of a node is

    1ip2(i),

    where the sum is over the classes i at the node, and p(i) is the observed fraction of classes with class i that reach the node. A node with just one class (a pure node) has Gini index 0; otherwise the Gini index is positive. So the Gini index is a measure of node impurity.

  • Deviance ('deviance') — With p(i) defined the same as for the Gini index, the deviance of a node is

    ip(i)logp(i).

    A pure node has deviance 0; otherwise, the deviance is positive.

  • Twoing rule ('twoing') — Twoing is not a purity measure of a node, but is a different measure for deciding how to split a node. Let L(i) denote the fraction of members of class i in the left child node after a split, and R(i) denote the fraction of members of class i in the right child node after a split. Choose the split criterion to maximize

    P(L)P(R)(i|L(i)R(i)|)2,

    where P(L) and P(R) are the fractions of observations that split to the left and right respectively. If the expression is large, the split made each child node purer. Similarly, if the expression is small, the split made each child node similar to each other, and hence similar to the parent node, and so the split did not increase node purity.

  • Node error — The node error is the fraction of misclassified classes at a node. If j is the class with the largest number of training samples at a node, the node error is

    1 – p(j).

Predictive Measure of Association

The predictive measure of association is a value that indicates the similarity between decision rules that split observations. Among all possible decision splits that are compared to the optimal split (found by growing the tree), the best surrogate decision split yields the maximum predictive measure of association. The second-best surrogate split has the second-largest predictive measure of association.

Suppose xj and xk are predictor variables j and k, respectively, and jk. At node t, the predictive measure of association between the optimal split xj < u and a surrogate split xk < v is

λjk=min(PL,PR)(1PLjLkPRjRk)min(PL,PR).

  • PL is the proportion of observations in node t, such that xj < u. The subscript L stands for the left child of node t.

  • PR is the proportion of observations in node t, such that xju. The subscript R stands for the right child of node t.

  • PLjLk is the proportion of observations at node t, such that xj < u and xk < v.

  • PRjRk is the proportion of observations at node t, such that xju and xkv.

  • Observations with missing values for xj or xk do not contribute to the proportion calculations.

λjk is a value in (–∞,1]. If λjk > 0, then xk < v is a worthwhile surrogate split for xj < u.

Algorithms

Element ma(i,j) is the predictive measure of association averaged over surrogate splits on predictor j for which predictor i is the optimal split predictor. This average is computed by summing positive values of the predictive measure of association over optimal splits on predictor i and surrogate splits on predictor j and dividing by the total number of optimal splits on predictor i, including splits for which the predictive measure of association between predictors i and j is negative.

Examples

expand all

Estimate the predictor importance for all variables in the Fisher iris data.

Load Fisher's iris data set.

load fisheriris

Grow an ensemble of 100 classification trees using AdaBoostM2.

ens = fitensemble(meas,species,'AdaBoostM2',100,'Tree');

Estimate the predictor importance for all predictor variables.

imp = predictorImportance(ens)
imp =

    0.0004    0.0016    0.1187    0.0403

The first two predictors are not very important in the ensemble.

Estimate the predictor importance for all variables in the Fisher iris data for an ensemble where the trees contain surrogate splits.

Load Fisher's iris data set.

load fisheriris

Grow an ensemble of 100 classification trees using AdaBoostM2. Specify to identify surrogate splits.

t = templateTree('Surrogate','on');
ens = fitensemble(meas,species,'AdaBoostM2',100,t);

Estimate the predictor importance and predictive measures of association for all predictor variables.

[imp,ma] = predictorImportance(ens)
imp =

    0.0674    0.0417    0.1582    0.1537


ma =

    1.0000         0         0         0
    0.0115    1.0000    0.0022    0.0054
    0.2963    0.1893    1.0000    0.5695
    0.0615    0.0317    0.1833    1.0000

The first two predictors show much more importance than the analysis in Estimate Predictor Importance.

Was this topic helpful?