Path: news.mathworks.com!not-for-mail
From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Out of Memory
Date: Sat, 4 Apr 2009 08:42:02 +0000 (UTC)
Organization: Boeing
Lines: 59
Message-ID: <gr76gq$5ap$1@fred.mathworks.com>
References: <gr5ega$t7f$1@fred.mathworks.com> <gr5k49$akq$1@fred.mathworks.com> <gr6cci$k50$1@fred.mathworks.com>
Reply-To: <HIDDEN>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1238834522 5465 172.30.248.37 (4 Apr 2009 08:42:02 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Sat, 4 Apr 2009 08:42:02 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 756104
Xref: news.mathworks.com comp.soft-sys.matlab:530253


"Derek O'Connor" <derekroconnor@eircom.net> wrote in message <gr6cci$k50$1@fred.mathworks.com>...
> 
> 3.  You say "in the C = operations(C) instruction you need a copy of C".  Are you sure of that? Can anyone verify that statement? Can you explain these  lines of code?
> 
> >> C = 2*C; % --- out-of-memory failure ---
> >> C = C+C; % --- out-of-memory failure ---
> >>for i = 1:n, for j = 1:n, C(i,j) = 2*C(i,j);end;end;       % --- works ok ---
> >>for i = 1:n, for j = 1:n, C(i,j) = C(i,j)+C(i,j);end;end;  % --- works ok ---
> 

MATLAB first creates the result of 2*C. This requires a matrix the same size of C, which is what the phrase "copy of C" actually meant. So that doubles the amount of storage needed, at least temporarily. Then, had it worked, the original C memory is freed and C is set to this new result. That is why your double loops do not run out of memory ... they do the operation in place. Same comments for the C+C equation and loop. The fact that your 2*C operation runs out of memory whereas your loops do not should be proof enough that a temp matrix the same size as C is used.

If you really need to do these simple operations on a very large matrix that is not sharing memory with another variable, you can do them inplace in a mex routine. e.g.

#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    mwSize i, numel;
    double *pr;
    if( nrhs != 1 || nlhs != 0 ) {
        mexErrMsgTxt("Need one input and no outputs.");
    }
    if( !mxIsDouble(prhs[0]) ) {
        mexErrMsgTxt("Input must be double.");
    }
    if( mxIsSparse(prhs[0]) ) {
        numel = *(mxGetJc(prhs[0])+ mxGetN(prhs[0]));
    } else {
        numel = mxGetNumberOfElements(prhs[0]);
    }
    pr = mxGetPr(prhs[0]);
    for( i=0; i<numel; i++ ) {
        pr[i] *= 2.0;
    }
}

To mex it just call the file mult2.c and then do this:

>> mex mult2.c

Here is a timing test of the three methods:

>> a=rand(5000);
>> tic;2*a;toc
Elapsed time is 0.458821 seconds.
>> tic;for k=1:5000;for m=1:5000; a(k,m) = 2*a(k,m);end;end;toc
Elapsed time is 43.492720 seconds.
>> tic;mult2(a);toc
Elapsed time is 0.134097 seconds.

> 5. "... and last, you really need a double random number? Try to recast to something less expensive."
> 
> Oh dear! I've been here before. See James Tursa's and my posts on this subject here:
> http://www.mathworks.com/matlabcentral/newsreader/view_thread/239186#637344
> 

Based on that thread, I have been experimenting with a sparse int8 class that I have created. So far it just does simple things like plus, minus, and times. I am slowly working on adding more capability for potential upload to the FEX someday. Would this be of interest to you? If so, what operations would you need/want for a sparse int8 class?

James Tursa