MATLAB Answers

Interpretation of plot: classification loss about number of features and number of observations

30 views (last 30 days)
asaad sellmann
asaad sellmann on 2 Oct 2020 at 6:09
Edited: asaad sellmann on 12 Oct 2020 at 13:26
Hi everyone,
I am having 2 classes and 30 Probands that each have "created" 6 observations, 3 in each class and I want to use fitctree for a "simple" classification algorithm.
To test the system I have created bar3 plots with N-Probands from 1 to 20 (6 to 120 observations for the training set) and N-Features from 1 to 10.
In a first step I test every single of my 41 features by training fitctree(xFeature) with one feature and calculate the loss of that model. That way I can rank my features and sort them by loss. choosing n features for the plot means n best features from that rank.
So my questiond here are how do I properly interpret my plots/ Am I making mistakes/ what kind of mistakes might I have made.
Especially: How do I explain the zeros for 4-6 probands? I really don't understand how it is possible to have so many zero-losses in that area of the plot
Plot when loss is calculated with
cvloss(tree, 'KFold', 5);
Plot when loss is calculated with
loss(tree, X, Y);
Thanks a lot in advance!
some additional infos:
Probands wear sensors from which I use the acceleration-data. From that data I am calculating a bunch of paramaters on each of the axis of each of the (3) sensors.

  1 Comment

asaad sellmann
asaad sellmann on 12 Oct 2020 at 13:26
Using the classification learner with 20 out of 30 Probands I get prediction accuracies of 95.8%. Exporting the model and then using the predction function with my remaining 10 probands leaves me with 95.1%
I am very uncertain if this is possible. Maybe you could give me some hints on how to test for overfitting. Or am I to suspicious and the fact that my probands were healthy subjects that were instructed right before the recording of the data leads to such a "good" dataset?
-----
after a few iterations with fewer features I even get to 100% prediciton accuracy with a quadratic svm..

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!