Predict response for observations not used for training
label = kfoldPredict(obj)
[label,score] = kfoldPredict(obj)
[label,score,cost] = kfoldPredict(obj)
class labels predicted by
label = kfoldPredict(
obj, a cross-validated
classification. For every fold,
class labels for in-fold observations using a model trained on out-of-fold
Vector of class labels of the same type as the response data used in training
Numeric matrix of size
Numeric matrix of misclassification costs of size
Find the cross-validation predictions for a model based on Fisher's iris data.
Load Fisher's iris data set.
Train an ensemble of classification trees using AdaBoostM2. Specify tree stumps as the weak learners.
rng(1); % For reproducibility t = templateTree('MaxNumSplits',1); Mdl = fitcensemble(meas,species,'Method','AdaBoostM2','Learners',t);
Cross validate the trained ensemble using 10-fold cross validation.
CVMdl = crossval(Mdl);
Estimate cross-validation predicted labels and scores.
[elabel,escore] = kfoldPredict(CVMdl);
Display the maximum and minimum scores of each class.
ans = 1×3 9.3862 8.9871 10.1866
ans = 1×3 0.0018 3.8359 0.9573
The average misclassification cost is the mean misclassification cost for predictions made by the cross-validated classifiers trained on out-of-fold observations. The matrix of expected costs per observation is defined in Cost.
For discriminant analysis, the score of a classification is the posterior probability of the classification. For the definition of posterior probability in discriminant analysis, see Posterior Probability.
For ensembles, a classification score represents the confidence of a classification into a class. The higher the score, the higher the confidence.
Different ensemble algorithms have different definitions for their scores. Furthermore, the range of scores depends on ensemble type. For example:
AdaBoostM1 scores range from –∞
Bag scores range from
For trees, the score of a classification of a leaf node is the posterior probability of the classification at that node. The posterior probability of the classification at a node is the number of training sequences that lead to that node with the classification, divided by the number of training sequences that lead to that node.
For example, consider classifying a predictor
Generate 100 random points and classify them:
rng(0,'twister') % for reproducibility X = rand(100,1); Y = (abs(X - .55) > .4); tree = fitctree(X,Y); view(tree,'Mode','Graph')
Prune the tree:
tree1 = prune(tree,'Level',1); view(tree1,'Mode','Graph')
The pruned tree correctly classifies observations that are less
than 0.15 as
true. It also correctly classifies
observations from .15 to .94 as
it incorrectly classifies observations that are greater than .94 as
Therefore, the score for observations that are greater than .15 should
be about .05/.85=.06 for
true, and about .8/.85=.94
Compute the prediction scores for the first 10 rows of
[~,score] = predict(tree1,X(1:10)); [score X(1:10,:)]
ans = 10×3 0.9059 0.0941 0.8147 0.9059 0.0941 0.9058 0 1.0000 0.1270 0.9059 0.0941 0.9134 0.9059 0.0941 0.6324 0 1.0000 0.0975 0.9059 0.0941 0.2785 0.9059 0.0941 0.5469 0.9059 0.0941 0.9575 0.9059 0.0941 0.9649
Indeed, every value of
X (the right-most
column) that is less than 0.15 has associated scores (the left and
center columns) of
while the other values of
X have associated scores
0.09. The difference
0.09 instead of the expected
is due to a statistical fluctuation: there are
X in the range
of the expected