CUDA_ERROR​_LAUNCH_TI​MEOUT

27 views (last 30 days)
María
María on 23 Nov 2011
Commented: Jakub Sikorowski on 10 Mar 2019
Hi
I am trying to execute PTX code, as the help file "Executing CUDA or PTX Code on the GPU" suggests.
I have executed the example successfully. But when I try to do the same with my own kernel, y retrieve this error:
??? Error using ==> feval
An error occurred during: retrieving information about free memory. The CUDA error was:
CUDA_ERROR_LAUNCH_TIMEOUT.
Any suggestions please?
Thanks,
María.

Accepted Answer

Konrad Malkowski
Konrad Malkowski on 30 Nov 2011
What type of cards do you have?
Do you have your user desktop extended to both graphics cards?
In general, this error occurs when a long running gpuArray or CUDA code is run on a GPU that is used for both graphics rendering and CUDA computations. The error is triggered by the Operating System, which limits the time that the GPU can dedicate to computations, vs. rendering the user desktop.
This limit does not apply to GPUs which do not have the desktop extended to them, such as standalone GPU accelerators, or cards running under TCC driver on Windows.
For this reason it is recommended that gpuArray and CUDAkernel evaluations be run on a GPU that is NOT attached to a display and does not have the Windows or Linux desktop extended onto it.
On the Windows Vista, Windows 7 operating systems individual GPU kernels running on GPUs that are responsible for desktop rendering are limited to a 2 second runtime.
If your system has GPUs without a display attached then you can manually modify the system configuration to disable the TDR mechanism. Disabling the TDR timeout will allow kernels to run for extended periods of time without triggering an error. In addition, on Windows, Tesla cards running under Tesla Compute Cluster drivers are not subject to this limitation.
For information on how to modify or disable the TDR timeout on Windows Vista and Windows 7 refer to:
  2 Comments
Jakub Sikorowski
Jakub Sikorowski on 10 Mar 2019
The microsoft link seems to be outdated. Setting up the timeout I have used Timeout Detection and Recovery (TDR) and Tdr registry keys.

Sign in to comment.

More Answers (4)

Edric Ellis
Edric Ellis on 23 Nov 2011
CUDA_ERROR_LAUNCH_TIMEOUT generally indicates that your kernel took too long to execute, and the operating system aborted the execution. Look at the output of running 'gpuDevice', and see whether KernelExecutionTimeout is true. If it is, then you may need to make changes to your system to get that value to be false. The changes you need to make depend on your operating system, and also whether you have a display attached to the GPU.

María
María on 23 Nov 2011
Thank you very much for your fast answer.
Yes, my KernelExecutionTimeout is true. My operating System is Windows 7 (64), I have two cards, one for visualization and another for computing, but I wordering if the computing card is also visualizing, how can I know it? I haven't intall the hardware...
Please, could you give a clue about how to fix it?

María
María on 7 Dec 2011
Hello,
I re-intalled the drivers, I included the TdrLevel=0 registry entry, and now I have KernelExecutionTimeout=0. But, the same problem with Matlab is happening... What annoys me is that I have the same code in CUDA-C launched with a C executable and it works fine, but Matlab continue saying:
??? Error using ==> feval An error occurred during: waiting for a kernel invocation to complete. The CUDA error was: CUDA_ERROR_UNKNOWN.__

María
María on 16 Feb 2012
I solve this updating the CUDA driver and toolkit to the latest version and updating my GPU driver.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!