Estimates of predictor importance
imp = predictorImportance(tree)
A row vector with the same number of elements as the number of predictors (columns) in tree.X. The entries are the estimates of predictor importance, with 0 representing the smallest possible importance.
predictorImportance computes estimates of predictor importance for tree by summing changes in the risk due to splits on every predictor and dividing the sum by the number of branch nodes. If tree is grown without surrogate splits, this sum is taken over best splits found at each branch node. If tree is grown with surrogate splits, this sum is taken over all splits at each branch node including surrogate splits. imp has one element for each input predictor in the data used to train tree. Predictor importance associated with this split is computed as the difference between the risk for the parent node and the total risk for the two children.
Estimates of predictor importance do not depend on the order of predictors if you use surrogate splits, but do depend on the order if you do not use surrogate splits.
If you use surrogate splits, predictorImportance computes estimates before the tree is reduced by pruning or merging leaves. If you do not use surrogate splits, predictorImportance computes estimates after the tree is reduced by pruning or merging leaves. Therefore, reducing the tree by pruning affects the predictor importance for a tree grown without surrogate splits, and does not affect the predictor importance for a tree grown with surrogate splits.
ClassificationTree splits nodes based on either impurity or node error. Impurity means one of several things, depending on your choice of the SplitCriterion name-value pair:
Gini's Diversity Index (gdi) — The Gini index of a node is
where the sum is over the classes i at the node, and p(i) is the observed fraction of classes with class i that reach the node. A node with just one class (a pure node) has Gini index 0; otherwise the Gini index is positive. So the Gini index is a measure of node impurity.
Deviance ('deviance') — With p(i) defined as for the Gini index, the deviance of a node is
A pure node has deviance 0; otherwise, the deviance is positive.
Twoing rule ('twoing') — Twoing is not a purity measure of a node, but is a different measure for deciding how to split a node. Let L(i) denote the fraction of members of class i in the left child node after a split, and R(i) denote the fraction of members of class i in the right child node after a split. Choose the split criterion to maximize
where P(L) and P(R) are the fractions of observations that split to the left and right respectively. If the expression is large, the split made each child node purer. Similarly, if the expression is small, the split made each child node similar to each other, and hence similar to the parent node, and so the split did not increase node purity.
Node error — The node error is the fraction of misclassified classes at a node. If j is the class with largest number of training samples at a node, the node error is
1 – p(j).
Estimate the predictor importance for all variables in the Fisher iris data:
load fisheriris tree = ClassificationTree.fit(meas,species); imp = predictorImportance(tree) imp = 0 0 0.0403 0.0303
The first two elements of imp are zero. Therefore, the first two predictors do not enter into tree calculations for classifying irises.
Estimate the predictor importance for all variables in the Fisher iris data for a tree grown with surrogate splits:
tree2 = ClassificationTree.fit(meas,species,... 'Surrogate','on'); imp2 = predictorImportance(tree2) imp2 = 0.0287 0.0136 0.0560 0.0556
In this case, all predictors have some importance. As you expect by comparing to the first example, the first two predictors are less important than the final two.
Estimates of predictor importance do not depend on the order of predictors if you use surrogate splits, but do depend on the order if you do not use surrogate splits. For example, permute the order of the data columns in the previous example:
load fisheriris meas3 = meas(:,[4 1 3 2]); tree3 = ClassificationTree.fit(meas3,species); imp3 = predictorImportance(tree2) imp3 = 0.0674 0 0.0033 0
The estimates of predictor importance are not a permutation of imp from the first example.
Estimate the predictor importance using surrogate splits.
tree4 = ClassificationTree.fit(meas3,species,... 'Surrogate','on'); imp4 = predictorImportance(tree4) imp4 = 0.0556 0.0287 0.0560 0.0136
imp4 is a permutation of imp2, demonstrating that estimates of predictor importance do not depend on the order of predictors with surrogate splits.