Clear Filters
Clear Filters

CUSOLVER_S​TATUS_INTE​RNAL_ERROR with RTX 4080 Super in MATLAB 2022a C MEX Function

16 views (last 30 days)
Hello,
I'm encountering a problem when running a CUDA-based C MEX function in MATLAB 2022a on a machine equipped with an RTX 4080 Super GPU. The specific error occurs when executing the line:
cusolverStatus_t status = cusolverDnCreate(&handle);
Here are the details:
  • The MEX function compiles successfully using the mexcuda command.
  • The same code operates correctly on this machine when using an RTX 2080Ti GPU with exactly the same software setup.
  • When the RTX 4080 Super is used, this error appears, even though the function runs without issues in Visual Studio, indicating that CUDA (Version 12.5) is properly configured on my system.
  • Upgrading to MATLAB 2024a resolves the issue, suggesting a compatibility problem with MATLAB 2022a. However, I am restricted to using MATLAB 2022a due to dependencies on other external software.
Additional context:
  • NVIDIA Driver Version: 555.99
  • CUDA Toolkit Version: 12.5
Has anyone else experienced similar issues with newer NVIDIA GPUs in MATLAB 2022a? Any insights or suggestions on how to resolve or circumvent this error with the RTX 4080 Super would be greatly appreciated.
Thank you!
  1 Comment
Umar
Umar on 21 Jun 2024
Hi Stefanos, It seems that the compatibility issue lies between MATLAB 2022a and the RTX 4080 Super GPU. Upgrading to MATLAB 2024a resolves the problem, indicating a version-specific conflict. Since you are constrained to MATLAB 2022a, consider checking for any MATLAB updates or patches that address GPU compatibility issues. Additionally, ensure that all CUDA libraries and drivers are up to date. You may also try adjusting compiler options or CUDA configurations specific to the RTX 4080 Super GPU. Seeking support from MathWorks or NVIDIA forums could provide further insights or workarounds tailored to your setup.

Sign in to comment.

Accepted Answer

Joss Knight
Joss Knight on 24 Jun 2024
Edited: Joss Knight on 25 Jun 2024
This is a known issue with CUDA 11.2, Ada cards, and cusolver. The solution is to upgrade MATLAB to R2023a or above, because it requires a new version of CUDA which cannot be provided in an update release.
It is possible that you can be walked through dropping a fixed version of cusparse and cusolver from CUDA 11.3, that would be compatible with MATLAB R2022a and not broken. However, it would require you to edit your MATLAB installation. (SPOILER: This turned out to be the only solution that worked.) Upgrading is definitely preferable.
Since this is a mex function, you have the option to compile using your own CUDA toolkit rather than the one shipped with MATLAB. To do this you will need to make some edits to mex options in order to allow mexcuda to use a version of the toolkit that doesn't match the version of MATLAB. I can walk you through this if this is your preferred route. At the moment mexcuda is not using the 12.5 tookit you installed because the version is wrong.
EDIT: Actually, I think I know how to force mexcuda to compile with your toolkit without any options file changes:
mexcuda -v CUDA_LIB_PATH='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\lib\x64' CUDA_ROOT='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5' LIBDEVICE='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\nvvm\libdevice' cusolver_handle_mex.cu
The -v will help you work out whether the correct version of nvcc was used. What I can't be sure of is that using CUDA 12.5 alongside CUDA 11.2 won't have unforeseen problems. If it does you might be better off installing CUDA 11.3 instead, which will be compatible with R2022a but won't have the cusolver bug.
  18 Comments
Stefanos
Stefanos on 26 Jun 2024
Yes, it would have been nice to not have to hack it this way but since it works it is a good solution for the time being until we can use a newer Matlab release. Thank you again!

Sign in to comment.

More Answers (1)

Joss Knight
Joss Knight on 24 Jun 2024

It's probably necessary for you to provide your code, since any number of things could have gone wrong prior to intializing cusolver which could prevent it from intializing successfully. For instance, did you start your mex function by calling mxGPUInit?

  2 Comments
Stefanos
Stefanos on 24 Jun 2024
I am not sure what mxGPUInit as I cannot seem to find any documentation on it. Could you point me to a page or reference? Also, it seems weird that the same code works on the 2022a version with another GPU or with the actual GPU I have the issue with the 2024a version. In addition, I have never had to use this previously mxGPUInit and have been working with MEX GPU code for quite a while.
Here is a simplified version of my MEX code that produces the error in question:
#include "mex.h"
#include <cusolverDn.h>
#include <cuda_runtime.h>
#include <cstdio>
// Function to convert cuSOLVER status to a string
const char *cusolverGetErrorString(cusolverStatus_t status)
{
switch (status)
{
case CUSOLVER_STATUS_SUCCESS:
return "CUSOLVER_STATUS_SUCCESS";
case CUSOLVER_STATUS_NOT_INITIALIZED:
return "CUSOLVER_STATUS_NOT_INITIALIZED";
case CUSOLVER_STATUS_ALLOC_FAILED:
return "CUSOLVER_STATUS_ALLOC_FAILED";
case CUSOLVER_STATUS_INVALID_VALUE:
return "CUSOLVER_STATUS_INVALID_VALUE";
case CUSOLVER_STATUS_ARCH_MISMATCH:
return "CUSOLVER_STATUS_ARCH_MISMATCH";
case CUSOLVER_STATUS_MAPPING_ERROR:
return "CUSOLVER_STATUS_MAPPING_ERROR";
case CUSOLVER_STATUS_EXECUTION_FAILED:
return "CUSOLVER_STATUS_EXECUTION_FAILED";
case CUSOLVER_STATUS_INTERNAL_ERROR:
return "CUSOLVER_STATUS_INTERNAL_ERROR";
case CUSOLVER_STATUS_MATRIX_TYPE_NOT_SUPPORTED:
return "CUSOLVER_STATUS_MATRIX_TYPE_NOT_SUPPORTED";
default:
return "Unknown cuSOLVER error";
}
}
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
// Print initial diagnostic information
mexPrintf("Starting cuSOLVER handle initialization.\n");
// Declare a cuSOLVER handle
cusolverDnHandle_t handle;
// Initialize the cuSOLVER handle
cusolverStatus_t status = cusolverDnCreate(&handle);
if (status != CUSOLVER_STATUS_SUCCESS)
{
mexErrMsgIdAndTxt("cusolver:initializationFailed",
"CUSOLVER initialization failed: %s", cusolverGetErrorString(status));
}
mexPrintf("cuSOLVER handle initialized successfully.\n");
// Destroy the cuSOLVER handle
status = cusolverDnDestroy(handle);
if (status != CUSOLVER_STATUS_SUCCESS)
{
mexErrMsgIdAndTxt("cusolver:destructionFailed",
"CUSOLVER destruction failed: %s", cusolverGetErrorString(status));
}
mexPrintf("cuSOLVER handle destroyed successfully.\n");
}
In addition here is the compilation command used:
% Set CUDA_PATH to the appropriate path
% setenv('MW_CUDA_FORWARD_COMPATIBILITY', '1');
setenv('CUDA_PATH', 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5');
% Add CUDA binaries to the system PATH
setenv('PATH', [getenv('CUDA_PATH') '\bin;' getenv('PATH')]);
% Compile the MEX function
mexcuda -v cusolver_handle_mex.cu ...
-L"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\lib\x64" ...
-I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.5\include" ...
-lcusolver -lcudart -lcublas

Sign in to comment.

Products


Release

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!