MATLAB Answers

0

How can I accelerate deep learning training using GPU?

Asked by Kyungsik Shin on 20 May 2019
Latest activity Commented on by NOSHEEN SOHAIL on 23 Oct 2019 at 8:13
I've made a simple neural network
It classifies MNIST handwritten digit using fully-connected layers
lgraph_2 = [ ...
imageInputLayer([28 28 1])
fullyConnectedLayer(512)
reluLayer
fullyConnectedLayer(256)
reluLayer
fullyConnectedLayer(128)
reluLayer
fullyConnectedLayer(10)
softmaxLayer
classificationLayer];
And the options in the neural network is
miniBatchSize = 10;
valFrequency = 5;
options = trainingOptions('sgdm', ...
'MiniBatchSize',miniBatchSize, ...
'MaxEpochs',5, ...
'InitialLearnRate',3e-4, ...
'Shuffle','every-epoch', ...
'ValidationData',augimdsValidation, ...
'ValidationFrequency',valFrequency, ...
'Verbose',true, ...
'Plots','training-progress', ...
'ExecutionEnvironment', 'parallel');
I expected when i use a GPU, it's training speed will be high
But when I train this network using Macbook(sigle CPU)
it takes 1 hour for around 2500 iterations
And when I use my desktop using RTX 2080Ti,
It takes much longer time to train.
MATLAB detects my GPU properly(I checked the GPU information using gpuDevice)
I don't know how can I accelerate the training proess.
Thank you in advance

  0 Comments

Sign in to comment.

2 Answers

Answer by Joss Knight
on 2 Jun 2019

Your mini-batch size is far too small. You're not going to get any benefit of GPU over CPU with that little GPU utilisation. Increase it to 512 or 1024, or higher (MNIST is a toy network - you could probably train the whole thing in a single mini-batch).
Also, the ExecutionEnvironment option you're looking for is gpu or auto, not parallel. parallel may be slowing things down in your case, if you have a second supported graphics card.

  0 Comments

Sign in to comment.


Answer by Shivam Sardana on 29 May 2019
Edited by Shivam Sardana on 29 May 2019

Considering CUDA® enabled NVIDIA® GPU with compute capability 3.0 or higher and Parallel Computing Toolbox™ are installed. Consider changing ‘ExecutionEnivronement’ to ‘gpu’. You can refer to the documentation link to see if this helps:
https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html

  1 Comment

I'm almost facing the same issue. my GeForce GTX1080 GPU does not show training progress, no update on a single iteration, single epoch even waited watching 3 days.. or it seems to be much slower than CPU training??
How is this happening, instead of being faster computation, it shows nothing. only big noisy sound from my GPU card is heard but no training plot progresses??
All requirements of CUDA® enabled NVIDIA® GPU with compute capability 3.0 or higher and Parallel Computing Toolbox™ are installed. Done changing ‘ExecutionEnivronement’ to ‘gpu
please help

Sign in to comment.