Clear Filters
Clear Filters

Why is the maximum variable size allowed by the program is exceeded when using Matlab Coder but not when running code directly?

8 views (last 30 days)
Dear all,
I have encountered an issue with using the Matlab Coder package. The Matlab code that I am trying to convert to C code works perfectly in the Matlab environment. However, when I try to convert it to C code using the Coder package I encounter the following error message:
Maximum variable size allowed by the program is exceeded.
I find this strange as the code works fine when run it as Matlab code. I have already set up Matlab to use the maximum amount of RAM available on my computer (16GB). Can anybody help me with this issue?
Thank you all in advance!
This is the code snippet where the error occurs. The final line of code is the part giving the issue.
Q0 = sparse(nbk*nbb*nbh*nbm*nbr,nbk*nbb*nbh*nbm*nbr);
for i=1:nbk
for j = 1:nbb
for h = 1:nbh
for m =1:nbm
for r_old = 1:nbr
Q0(i+nbk*(j-1)+nbk*nbb*(h-1)+nbk*nbb*nbh*(m-1)+nbk*nbb*nbh*nbm*(r_old-1),k_index(i,j,h,m,r_old,z)+nbk*(b_index(i,j,h,m,r_old,z)-1)+nbk*nbb*(h_index(i,j,h,m,r_old,z)-1)+nbk*nbb*nbh*(m_index(i,j,h,m,r_old,z)-1)+nbk*nbb*nbh*nbm*(r_index(i,j,h,m,r_old,z)-1)) = 1;
end
end
end
end
end
Q((z-1)*nbk*nbb*nbh*nbm*nbr+1:z*nbk*nbb*nbh*nbm*nbr,:) = kron(PI(z,:),Q0);
end
As = sparse(speye(nbk*nbz*nbb*nbh*nbm*nbr)-beta*Q);
  2 Comments
Steven Lord
Steven Lord on 21 Apr 2023
Where do the variables nbk, nbz, nbb, nbh, nbm, and nbr come from? Are they hard-coded in your code, are they passed into your function as input arguments, are they computed from input arguments?
What are typical values for those variables? If they're all 2 then your sparse matrices aren't going to be that large; if they're all 100 or 1000 they're going to be much, much larger.
What sizes are beta and Q?
What's the density of beta*Q? Is it possible that while speye is sparsely populated that enough of the off-diagonal elements of beta*Q are non-zero so the result is not sparsely populated? Consider, as an extreme example:
A = ones(10);
B = sparse(A);
whos A B
Name Size Bytes Class Attributes A 10x10 800 double B 10x10 1688 double sparse
B is stored as a sparse matrix but it's not sparsely populated since every element is non-zero. So the sparse storage actually consumes more memory than the full storage due to having to store the indices.
Steven Hoekstra
Steven Hoekstra on 24 Apr 2023
Edited: Steven Hoekstra on 24 Apr 2023
nbk, nbz, nbb, nbh, nbm, and nbr are all constants in the range from 2-10. These are fixed during the programme.
beta is a number usually between 0.9 and 0.99.
Q is a sparse Markovian transition matrix with nbz doubles on each row. It is created by taking the Kronecker product of a dense Markovian transition matrix PI with another transition matrix Q0 which has the density of an identity matrix ( one nonzero entry per row).
I have found a strange fix for the problem. If I replace the line which fills in matrix Q with the following line,
Q((z-1)*nbk*nbb*nbh*nbm*nbr+1:z*nbk*nbb*nbh*nbm*nbr,:) = beta*kron(PI(z,:),Q0);
and replace the line that defines As with the following line
As = sparse(speye(nbk*nbz*nbb*nbh*nbm*nbr)-Q);
the problem is solved. The workings of the algorithm do not change.
However, I still do not understand why this "fix" works. I would like to change the situation back to the old situation as this way of solving the problem changes the definition of my variables and obfusciates the meaning of some steps.
The complete code that works is pasted below.
Q0 = sparse(nbk*nbb*nbh*nbm*nbr,nbk*nbb*nbh*nbm*nbr);
for i=1:nbk
for j = 1:nbb
for h = 1:nbh
for m =1:nbm
for r_old = 1:nbr
Q0(i+nbk*(j-1)+nbk*nbb*(h-1)+nbk*nbb*nbh*(m-1)+nbk*nbb*nbh*nbm*(r_old-1),k_index(i,j,h,m,r_old,z)+nbk*(b_index(i,j,h,m,r_old,z)-1)+nbk*nbb*(h_index(i,j,h,m,r_old,z)-1)+nbk*nbb*nbh*(m_index(i,j,h,m,r_old,z)-1)+nbk*nbb*nbh*nbm*(r_index(i,j,h,m,r_old,z)-1)) = 1;
end
end
end
end
end
Q((z-1)*nbk*nbb*nbh*nbm*nbr+1:z*nbk*nbb*nbh*nbm*nbr,:) = beta*kron(PI(z,:),Q0);
end
As = sparse(speye(nbk*nbz*nbb*nbh*nbm*nbr)-Q);

Sign in to comment.

Answers (1)

Gowthami
Gowthami on 24 Apr 2023
Hello Steven,
MATLAB will attempt to create a matrix with a large number of elements, as long as that number of elements is less than the maximum number of elements allowed in a matrix.
You are attempting to create a matrix with more elements than the maximum number of elements allowed in MATLAB.
If the matrix you are attempting to create has relatively few nonzero elements, you may be able to create it as a sparse matrix. You can use the SPARSE function and the other sparse matrix manipulation functions to create and manage this matrix. For more information on SPARSE function, type the following command in command window for a list of the sparse matrix manipulation functions.
help sparfun
However, that the limitation on the maximum number of elements still exists, but now it only applies to the nonzero elements of the sparse matrix. If your matrix is not sparse, however, you will need to break it into sections with a number of elements less than the maximum returned by the COMPUTER function.
You can determine this maximum using the COMPUTER function in the following way -
[str, maxsize] = computer;
  1 Comment
Steven Hoekstra
Steven Hoekstra on 24 Apr 2023
The amount of elements in the matrix is less than the maximum amount of elements that are allowed in Matlab. Moreover, the code runs when run as Matlab code without any erros. However the code cannot be converted to C code as then Matlab coder gives the error. It is the conversion from Matlab code into C code which gives the error.
When reordering the assignment of the variables the error disappears hence it cannot be the size of the matrix. Additionally, even when the matrix is reduced to a small matrix the error keeps occuring.

Sign in to comment.

Categories

Find more on Data Type Conversion in Help Center and File Exchange

Products


Release

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!