How to plot test and validation accuracy every epoch using Computer vision system toolbox? And what about overfitting?

85 views (last 30 days)
I am finding many blogs of CNN and its related classification strategy in Matlab but I couldn't find how I could actually plot the validation set accuracy for every epoch along with the training set accuracy.
above documentation shows that I could plot training accuracy every epoch but not the validation set accuracy.
If that is not possible how to make sure that my network is not overfitting?

Accepted Answer

Sebastian Kammer
Sebastian Kammer on 26 May 2017
Hi Jay,
There is no option to display the validation set accuracy. However this should not be necessary. Functions in the Neural Network Toolbox employ a technique called Early Stopping, which takes the validation set error into consideration to prevent overfitting. The article I am linking to provides some nice discussions on overfitting, I would recommend you to read the other parts as well.
Cheers,
Sebastian
  5 Comments

Sign in to comment.

More Answers (2)

Evan Koester
Evan Koester on 27 Mar 2018
Edited: Evan Koester on 27 Mar 2018
This problem has been addressed in the newly released MATLAB 2018a. If you have a groundtruth of your data, you can load it as a pixelLabelImageDatastore. This can be used in your ValidationData trainingOptions.
An example:
val4data = imageDatastore(location of image data); val4label = load('location of label groundtruth');
val4label = pixelLabelDatastore(val4label.gTruth); val4gt = pixelLabelImageDatastore(val4data,val4label);
opts = trainingOptions('sgdm', ... 'MaxEpochs', 5000, ... 'InitialLearnRate', .05, ... 'VerboseFrequency',validationFrequency,... 'ValidationData',val4gt,... 'ValidationFrequency',5,... 'Plots','training-progress',... 'CheckpointPath', tempdir,... 'MiniBatchSize', 48);
This will plot the validation data loss on the same plot as training loss when training your CNN. In my application I used pixel-wise labeling. Prior to MATLAB version 2018a, there was not a way to perform this without making a checkpoint, test validation data, continue training type of algorithm as mentioned above.

Saira
Saira on 15 Jun 2020
Hi,
I have 5600 training images. I have extracted features using Principal Component Analysis (PCA). Then I am applying CNN on extracted features. My training accuracy is 30%. How to increase training accuracy?
Feature column vector size: 640*1
My training code:
% Convolutional neural network architecture
layers = [
imageInputLayer([1 640 1]);
reluLayer
fullyConnectedLayer(7);
softmaxLayer();
classificationLayer()];
options = trainingOptions('sgdm', 'Momentum',0.95, 'InitialLearnRate',0.0001, 'L2Regularization', 1e-4, 'MaxEpochs',5000, 'MiniBatchSize',8192, 'Verbose', true);

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!