GPU out of memory

Hi.

I am working with applying one of the MATLAB neural network examples to a data set that I have. My operating system is Windows !0. When I run the program on the CPU there are no errors. However, when I run it on the GPU I am getting an error:

"Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'. If the problem persists, reset the GPU by calling 'gpuDevice(1)'."

Error using trainNetwork (line 170)
GPU out of memory. Try reducing 'MiniBatchSize' using the trainingOptions function.

Error in Untitled (line 36)
convnet = trainNetwork(imds,layers,options);

Caused by:
    Error using nnet.internal.cnngpu.batchNormalizationForwardTrain
    Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'. If
    the problem persists, reset the GPU by calling 'gpuDevice(1)'.

My Code As Follows:

clc
clear 
close all
close all hidden;
[file1,path1]=uigetfile('*.*');
rgb=imread([path1,file1]);
figure,imshow(rgb);title('Input image');
rgb=imresize(rgb,[512 734]);

matlabroot = 'E:\project files';
digitDatasetPath = fullfile(matlabroot,'Dataset');
imds = imageDatastore(digitDatasetPath,'IncludeSubfolders',true,'LabelSource','foldernames');

layers = [
    imageInputLayer([512 734 3])
    convolution2dLayer(3,32,'Stride',1,'Padding','same','Name','conv_1')
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(2,'Stride',2,'Name','maxpool_1')
    
    convolution2dLayer(3,64,'Stride',1,'Padding','same','Name','conv_2')
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(2,'Stride',2,'Name','maxpool_2')
   
    convolution2dLayer(3,128,'Stride',1,'Padding','same','Name','conv_3')
    batchNormalizationLayer
    reluLayer
    maxPooling2dLayer(2,'Stride',2,'Name','maxpool_3')
    
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];

options = trainingOptions('sgdm','Plots','training-progress','MaxEpochs',10,'initialLearnRate',0.001);
convnet = trainNetwork(imds,layers,options);
YPred = classify(convnet,rgb);
output=char(YPred);


if output=='1'
    msgbox('No tumor is detected')
else
    msgbox('Tumor is detected')
end

my GPU Device Details are as follows

gpuDevice

ans = 

  CUDADevice with properties:

                      Name: 'GeForce GTX 1650'
                     Index: 1
         ComputeCapability: '7.5'
            SupportsDouble: 1
             DriverVersion: 10.2000
            ToolkitVersion: 10.1000
        MaxThreadsPerBlock: 1024
          MaxShmemPerBlock: 49152
        MaxThreadBlockSize: [1024 1024 64]
               MaxGridSize: [2.1475e+09 65535 65535]
                 SIMDWidth: 32
               TotalMemory: 4.2950e+09
           AvailableMemory: 3.0421e+09
       MultiprocessorCount: 16
              ClockRateKHz: 1560000
               ComputeMode: 'Default'
      GPUOverlapsTransfers: 1
    KernelExecutionTimeout: 1
          CanMapHostMemory: 1
           DeviceSupported: 1
            DeviceSelected: 1

[Joss Knight]

What is your question?

[Reddyprasad Vankireddy]

How to solve this GPU out of memory issue.. i tried previous answers in the community but I didn't work.

Answer 1

[Joss Knight]

In your example code you are using the default mini-batch size of 128. Reduce the MiniBatchSize training option until you stop getting the out of memory error.

[Ilya Kuprov]

Joss, there are still some weird memory leaks on GPU - we commonly see GPU memory stuck as used even after a function that used GPU arrays exits. It gets really bad in parallel pools where even baseline memory utilisation from sending one byte to GPU on each worker can down it. Clearing variables and even reassigning them to empty arrays does not help. We would much appreciate some GPU memory management functionality or better background grabage collection.

[Ilya Kuprov]

To elaborate:

spmd

a=gpuArray(1);

end

clear('a');

and watch GPU memory usage in the Task Manager.

[Joss Knight]

MATLAB holds onto GPU memory once allocated because memory allocation is a huge bottleneck for performance. All the Deep Learning frameworks do this. You will find that memory is pooled up to a quarter of the total memory.

If you have to release GPU memory then you can reset the device using reset(gpuDevice) or unselect it using gpuDevice([]). Make sure you've gathered any variables you need to hold onto first. If you need to share a GPU between parallel workers you can use this, however sharing a GPU cannot give you GPU parallelism (the GPU driver serialises operations from different processes) so you'd be better off trying to vectorize your code where possible.

Answer 2

[AKHTAR JAMIL]

Late though it might help, i faced same situation and set minibatch even to 1 but didnot work at all with GPU, so i switched to CPU by adding 'ExecutionEnvironment', 'cpu'

options = trainingOptions('sgdm','Plots','training-progress','MaxEpochs',10,'initialLearnRate',0.001, 'ExecutionEnvironment', 'cpu');

[Meriem Serdi]

I did the same thing as you and I am waiting for my program for about 90 minute, I am training with 3000 images. You had the same problem ?

[Afiq Amirudin]

had exact same problem, try changing to CPU will take the training forever man even my training only about 1200 images, let me know if you already solve it

[Kuno Bruswachtl]

still have the same problem here.

[Van Vy]

I reduce minibatchsize to 1 and change ExecutionEnvironment= cpu but the problem is still same.

But I reduce the outputsize in fullyConnectedLayer() and I pass this problem.

[Anisa Azizah]

i have the same problem. how to reduce the outputsize in fullyConnectedLayer Mr. Van ?