How to deal with a huge matrix?

13 views (last 30 days)
Martin
Martin on 18 Jul 2013
Hi!
I'm dealing with a huge matrix: to give a little context if it makes sense, it is a transition matrix of a MDP, I thus have to calculate first all the transition probabilities (==> many loops) and then use this matrix in Bellman's equation (==> many matrices products)
In clear, I must create a big matrix (~1GB, potentially more) in my program before using it several times.
My question is, should I rather:
1 - divide my program in several parts (one little "mother function" calling the calculation function, getting the big matrix back, and launching the second function that uses it), which is generally advised?
2 - or put all in the same function in an ugly way :)
Because I'm afraid Matlab will make many copies of the matrix with the first solution. If the total size used by Matlab gets bigger than my RAM, it could cause troubles.
Thanks!
Martin

Accepted Answer

Matt J
Matt J on 18 Jul 2013
Edited: Matt J on 18 Jul 2013
Because I'm afraid Matlab will make many copies of the matrix with the first solution.
MATLAB never copies a matrix as it gets passed around to different functions unless a change is made to the matrix by one of the functions.
As for the size of the matrix itself, have you considered constructing it using the SPARSE command? Or is the matrix inherently dense?
  4 Comments
Matt J
Matt J on 18 Jul 2013
Edited: Matt J on 19 Jul 2013
% function g function g(m); m(2,4) = 56;
This is the only point in what you've shown where a copy of m would be made. Here's some further relevant reading:
Basically, for the most part, MATLAB is smart enough never to make copies unless it needs them. Applying RESHAPE to a matrix also doesn't create copies.
Extracting a sub-matrix by indexing, however, does allocate memory for the sub-matrix, unless the indexing is done on the left hand side of an assignment. As an example, B(1:4) is allocated memory below
A=rand(5);
B=1:8;
C=A(B(1:4),:); %The submatrix of B here occupies its own block of memory.
Conversely, the following
B(:)=1;
does not result in any new memory allocation for the already existing B. The values of B simply get over-written in place. In
B(1:4)=2;
memory might be allocated for the index vector 1:4 itself (depending on your MATLAB version) but not for the altered B.
Martin
Martin on 19 Jul 2013
Thank you for your answer.

Sign in to comment.

More Answers (0)

Categories

Find more on Operators and Elementary Operations in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!