Learn more about Statistics Toolbox

Classification Trees

On this page…
Introduction Example: Classification Trees References

Introduction

Parametric models specify the form of the relationship between predictors and a response, as in the Hougen-Watson model described in Parametric Models. In many cases, however, the form of the relationship is unknown, and a parametric model requires assumptions and simplifications. Regression Trees offer a nonparametric alternative. When response data are categorical, classification trees are a natural modification.

Note   This section demonstrates methods for objects of the classregtree class. These methods supersede the functions treefit, treedisp, treeval, treeprune, and treetest, which are maintained in Statistics Toolbox software only for backwards compatibility.

Back to Top

Example: Classification Trees

This example uses Fisher's iris data in fisheriris.mat to create a classification tree for predicting species using measurements of sepal length, sepal width, petal length, and petal width as predictors. Note that, in this case, the predictors are continuous and the response is categorical.

1. Load the data and use the classregtree constructor of the classregtree class to create the classification tree:

   load fisheriris
   
   t = classregtree(meas,species,...
                    'names',{'SL' 'SW' 'PL' 'PW'})
   t = 
   Decision tree for classification
   1  if PL<2.45 then node 2 else node 3
   2  class = setosa
   3  if PW<1.75 then node 4 else node 5
   4  if PL<4.95 then node 6 else node 7
   5  class = virginica
   6  if PW<1.65 then node 8 else node 9
   7  class = virginica
   8  class = versicolor
   9  class = virginica

   t is a classregtree object and can be operated on with any of the methods of the class.

2. Use the type method of the classregtree class to show the type of the tree:

   treetype = type(t)
   treetype =
   classification
   

   classregtree creates a classification tree because species is a cell array of strings, and the response is assumed to be categorical.

3. To view the tree, use the view method of the classregtree class:

   view(t)

   

   The tree predicts the response values at the circular leaf nodes based on a series of questions about the iris at the triangular branching nodes. A true answer to any question follows the branch to the left; a false follows the branch to the right.

4. The tree does not use sepal measurements for predicting species. These can go unmeasured in new data, and you can enter them as NaN values for predictions. For example, to use the tree to predict the species of an iris with petal length 4.8 and petal width 1.6, type:

   predicted = t([NaN NaN 4.8 1.6])
   predicted = 
       'versicolor'

   Note that the object allows for functional evaluation, of the form t(X). This is a shorthand way of calling the eval method of the classregtree class. The predicted species is the left leaf node at the bottom of the tree in the previous view.

5. You can use a variety of other methods of the classregtree class, such as cutvar and cuttype to get more information about the split at node 6 that makes the final distinction between versicolor and virginica:

   var6 = cutvar(t,6) % What variable determines the split?
   var6 = 
       'PW'
   
   type6 = cuttype(t,6) % What type of split is it?
   type6 = 
       'continuous'
   

6. Classification trees fit the original (training) data well, but may do a poor job of classifying new values. Lower branches, especially, may be strongly affected by outliers. A simpler tree often avoids overfitting. You can use the prune method of the classregtree class to find the next largest tree from an optimal pruning sequence:

   pruned = prune(t,'level',1)
   pruned = 
   Decision tree for classification
   1  if PL<2.45 then node 2 else node 3
   2  class = setosa
   3  if PW<1.75 then node 4 else node 5
   4  if PL<4.95 then node 6 else node 7
   5  class = virginica
   6  class = versicolor
   7  class = virginica
   
   view(pruned)

   

To find the best classification tree, employing the techniques of resubstitution and cross-validation, use the test method of the classregtree class.

Back to Top

References

[1] Breiman, L., et al., Classification and Regression Trees, Chapman & Hall, Boca Raton, 1993.

Back to Top

Naive Bayes Classification

Regression and Classification by Bagging Decision Trees

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS