Path: news.mathworks.com!not-for-mail
From: "heidi jiang" <heydee@gmail.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: faster way to implement corr
Date: Wed, 16 May 2012 17:06:15 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 26
Message-ID: <jp0mq7$snn$1@newscl01ah.mathworks.com>
References: <jourtj$quo$1@newscl01ah.mathworks.com> <jov0c3$ebk$1@newscl01ah.mathworks.com> <jov1c9$i10$1@newscl01ah.mathworks.com> <jov443$snv$1@newscl01ah.mathworks.com> <jovb54$q02$1@newscl01ah.mathworks.com>
Reply-To: "heidi jiang" <heydee@gmail.com>
NNTP-Posting-Host: www-02-blr.mathworks.com
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: newscl01ah.mathworks.com 1337187975 29431 172.30.248.47 (16 May 2012 17:06:15 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Wed, 16 May 2012 17:06:15 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 3488832
Xref: news.mathworks.com comp.soft-sys.matlab:768052

"Roger Stafford" wrote in message <jovb54$q02$1@newscl01ah.mathworks.com>...
> "Roger Stafford" wrote in message <jov443$snv$1@newscl01ah.mathworks.com>...
> >  This could be done with a large set of vectors as a matrix multiplication, but I suspect you would have memory problems with a 230000 by 230000 matrix, and would at least have to do such a multiplication piecemeal.
> - - - - - - - - -
>   There is one point I want to correct in my previous discussion.  I mentioned a 230000 by 230000 matrix, and that is surely incorrect.  Each vector undoubtedly has far fewer than 230000 elements.
> 
>   If each of your vectors has n elements, then let A be a 230000 by n matrix in which the rows are the modified vectors and for which you hopefully have enough memory.  Then I envision an iteration that takes m of the vectors at a time:
> 
>  for k = 1:230000/m % m a factor of 230000
>   C = A*A((k-1)*m+1:k*m,:).'; % <-- Correlations
>   % C will be 230000 by m in size
>   % Each column has correlations of all vectors w.r. to
>   %    one of the vectors (k-1)*m+1 to k*m.
>   % Apply nonlinear ops. on each column of C to get "average" correlation
>   % Result is m "average" correlations which should be stored
>  end
> 
>   Here you would make m as large as possible and still be able to store a 230000 by m matrix C.  Each trip through the for-loop would encompass what m of your iterations accomplished and 'corr' will not have been called.
> 
>   I hope all this makes sense.
> 
> Roger Stafford

Hi,

This makes sense and I'll try it. Thank you!