CUDA Error: Illegal Memory Access in MATLAB Code

Hi everyone,

I'm encountering a CUDA error in my MATLAB code during the second iteration of a loop. Specifically, the error occurs at the wait(gd); command after calling setConstantMemory(kPAM, 'g_nT_Start', single((i) * Max_T_dim));. The error message is:

Error using parallel.gpu.CUDADevice/wait

Encountered unexpected error during CUDA execution. The CUDA error was:

an illegal memory access was encountered

Here is the relevant part of my code:

kPAM = parallel.gpu.CUDAKernel('PAM.ptx', 'PAM.cu', '_PAM');

kPAM.ThreadBlockSize = [1024,1,1];

kPAM.GridSize = [ceil(struct_GPU.Pix.nTdim/kPAM.ThreadBlockSize(1)),struct_GPU.Pix.nYdim*struct_GPU.Pix.nXdim, struct_GPU.Pix.nZdim];

setConstantMemory(kPAM,'g_nT_Start',single(0));

for i = 1:Lteration

if i == Lteration % Last iteration

last_T_dim = struct_GPU.Pix.nTdim - (Max_T_dim * i);

last_input_Array = gpuArray(single(zeros(last_T_dim, struct_GPU.Pix.nZdim, ...

struct_GPU.Pix.nXdim * struct_GPU.Pix.nYdim)));

assignin('base', 'PAM_txyz_gpu', last_input_Array);

assignin('base', 'PAM_txyz_cpu', last_input_Array);

elseif i > 1

PAM_txyz_gpu = input_Array;

PAM_txyz_cpu = output_Array;

else

PAM_txyz_gpu = input_Array;

PAM_txyz_cpu = output_Array;

end

[PAM_txyz_gpu] = feval(kPAM, PAM_txyz_gpu, RFData_gpu);

setConstantMemory(kPAM, 'g_nT_Start', single((i) * Max_T_dim));

wait(gd);

PAM_txyz_cpu = gather(PAM_txyz_gpu);

wait(gd);

PAM_xyz_cpu = PAM_xyz_cpu + squeeze(sum(PAM_txyz_cpu, 1));

end

clear PAM_txyz_cpu PAM_txyz_gpu last_input_Array input_Array;

reset(gd);

Does anyone have any suggestions on how to resolve this illegal memory access issue?

Thanks in advance!

Answer 1

[Shivani]

Hello @轶凡

If you are using a MATLAB version earlier than MATLAB R2017a, then it is likely that you are encountering this error because the Pascal architecture cards are not fully supported by the CUDA toolkit in previous releases. Kindly refer to the following External Bug Report link for more information on possible workarounds to this issue: https://www.mathworks.com/support/bugreports/1439741

Additionally, you can refer to the following MATLAB Answer thread for more details:

• https://www.mathworks.com/matlabcentral/answers/303228-cuda_error_illegal_address
• https://www.mathworks.com/matlabcentral/answers/309235-can-i-use-my-nvidia-pascal-architecture-gpu-with-matlab-for-gpu-computing

If you are using a later version of MATLAB and still encountering this error, it is because your code is attempting to read from or write to a memory location that it shouldn't. Ensure that the value being set does not cause the kernel to access out-of-bounds memory. For example, if "(i) * Max_T_dim" results in an index beyond what your kernel expects for the last iteration, this could lead to an illegal memory access.

Hope this helps!

[轶凡]

Thank you very much for your response and help. I am using version 2022b. After rechecking the code, I found that the line setConstantMemory(kPAM, 'g_nT_Start', single((i) * Max_T_dim)); was problematic. It turns out that once constant memory is set, it cannot be changed in subsequent iterations. I have now modified the code to change the operations that require modifying constant memory to instead modify a gpuArray. I found that it runs correctly now. Thanks again for your assistance！