"Out of memory" error when enabling validation during the process of training a neural network

1 view (last 30 days)
I'm using Matlab R2019a with the Deep Learning Toolbox and implementing my own algorithm following the structure of this tutorial.
Now the problem is, when I use my own dataset, the validation set contains about 2,000 images, which makes it not possible to load all of them into memory. Therefore, if I enable validation when using trainNetwork(), my system will prompt an error saying "Out of memory" even the batch size is set to 8. If validation is disabled, the training process works perfectly when the batch size is 32.
My computer configuration is:
  • Intel 8700K.
  • Nvidia GTX 1080TI
  • 32G memory
I searched for the documentation of trainingOptions and there is only one parameter called "miniBatchSize". I'm wondering whether that works for validation as well or just for training (According to my experience it seems to be the latter case). If the parameter only works for training, is there a way to perform validation using a batch processing manner (such as providing our own testing function in between as callbacks)?
Thank you.

Answers (0)

Categories

Find more on Deep Learning Toolbox in Help Center and File Exchange

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!