Why Matlab recognizes but doesn't use the GPU? How to fix it?

I am currently using Matlab R2018b. I have a Nvidia Quadro P5000 on my machine. Matlab recognizes the GPU by returning the following for "gpuDevice":

Name: 'Quadro P5000'
Index: 1
ComputeCapability: '6.1'
SupportsDouble: 1
DriverVersion: 10.1000
ToolkitVersion: 9.1000
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 1.7180e+10
AvailableMemory: 1.3074e+10
MultiprocessorCount: 20
ClockRateKHz: 1733500
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1

But when I run the codes with gpu functions, they are not run on the gpu (I also tried it with DriverVersion: 9.1 and ToolkitVersion: 9.1 with no success).

This is the output of nvidia-smi:

+-----------------------------------------------------------------------------+ 
| NVIDIA-SMI 431.70       Driver Version: 431.70       CUDA Version: 10.1     | 
|-------------------------------+----------------------+----------------------+ 
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC | 
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. | 
|===============================+======================+======================| 
|   0  Quadro P5000       WDDM  | 00000000:17:00.0  On |                  Off | 
| 30%   49C    P0    44W / 180W |   3035MiB / 16384MiB |      0%      Default | 
+-------------------------------+----------------------+----------------------+ 
                                                                                
+-----------------------------------------------------------------------------+ 
| Processes:                                                       GPU Memory | 
|  GPU       PID   Type   Process name                             Usage      | 
|=============================================================================| 
|    0      1076    C+G   ...iles\MATLAB\R2018b\bin\win64\MATLAB.exe N/A      | 
|    0      1344    C+G   Insufficient Permissions                   N/A      | 
|    0      1604    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0      3424    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0      3508    C+G   ...t_cw5n1h2txyewy\ShellExperienceHost.exe N/A      | 
|    0      9016    C+G   ...oftEdge_8wekyb3d8bbwe\MicrosoftEdge.exe N/A      | 
|    0      9120    C+G   C:\Windows\explorer.exe                    N/A      | 
|    0      9356    C+G   ...dows.Cortana_cw5n1h2txyewy\SearchUI.exe N/A      | 
|    0     10004    C+G   ...50.38.0_x64__kzf8qxf38zg5c\SkypeApp.exe N/A      | 
|    0     10812    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     11468    C+G   ...dqpraam7r\AcrobatNotificationClient.exe N/A      | 
|    0     11488    C+G   ...11411.0_x64__8wekyb3d8bbwe\Video.UI.exe N/A      | 
|    0     13476    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     13744    C+G   ...sktop App\AcWebBrowser\AcWebBrowser.exe N/A      | 
|    0     14080    C+G   ...m\win64\chromium\jxbrowser-chromium.exe N/A      | 
|    0     14116    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     14144    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     15084    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     15108    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     16124    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
|    0     16160    C+G   ...tEdge_8wekyb3d8bbwe\MicrosoftEdgeCP.exe N/A      | 
+-----------------------------------------------------------------------------+ 

and here is the output of nvcc --version in the cmd:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Mon_Jan__9_17:32:33_CST_2017
Cuda compilation tools, release 8.0, V8.0.60

How can I make Matlab use the GPU for my image processing and DL codes?

Thanks.

[Matt J]

What Matlab code are you running, and what evidence do you see that the GPU is not being used?

[Amirtaha Taebi]

Thanks Matt! I tried both codes written by me (where I transfered arrays to gpu using gpuArray, etc.) as well as Matlab examples such as:

openExample('nnet/VisualizeConvolutionalNeuralNetworkLayersExample')

About the evidences:

1. When I check the GPU usage in the windows task manager, GPU is fully loaded. The GPU usage is usually below 3%.
2. The computation time is similar to when I use cpu (for example, when I transfer the data to gpu using gpuArray vs when I use cpu).

Thanks.

[Matt J]

If the GPU is "fully loaded", wouldn't that indicate that it is being used?

It would be easiest if you display your gpuArray test code here, so that we can study exactly what operations you are timing/comparing and how.

[Amirtaha Taebi]

I’m sorry Matt! There was a typo in my comment. The GPU is NOT fully loaded actually. This happens when I run the Matlab example that I shared above. Do you have any thoughts why this happens? Thanks.

[Joss Knight]

Try using GPUBench to test the speed of your GPU, to give you an idea of what you should expect. I'm not sure if the Windows Task Manager shows compute utilisation, I think it just shows graphics operations.

[Joss Knight]

You may need to select some special monitoring to show CUDA usage.

[Amirtaha Taebi]

Hi Joss, I used GPUBench. It seems that my GPU actually works. The GPU usage (in windows task manager) went also all the up to 100% during the tests. So, I'm not sure why it doesn't work with my code. There should be some code debugging needed. Thanks,

[Joss Knight]

Utilisation is an odd thing, and I'm not sure how the Task Manager displays it. Moreover, you indicate you are running some code that uses deepDreamImage, but is also doing a variety of other computation. Deep dream contains a mixture of CPU and GPU computation of different levels of compute intensity, and this may not be the best way to showcase your GPU performance. Try something like Transfer Learning Using GoogleNet to see if you can get better-looking numbers.