Path: news.mathworks.com!not-for-mail
From: "Ashish Uthama" <first.last@mathworks.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Distributed Computing
Date: Sun, 15 Mar 2009 17:16:58 -0400
Organization: TMW
Lines: 76
Message-ID: <op.uquqqkcpa5ziv5@uthamaa.dhcp.mathworks.com>
References: <gpjdp5$5cd$1@fred.mathworks.com>
NNTP-Posting-Host: uthamaa.dhcp.mathworks.com
Mime-Version: 1.0
Content-Type: text/plain; format=flowed; delsp=yes; charset=iso-8859-15
Content-Transfer-Encoding: 7bit
X-Trace: fred.mathworks.com 1237151818 19183 172.31.57.126 (15 Mar 2009 21:16:58 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Sun, 15 Mar 2009 21:16:58 +0000 (UTC)
User-Agent: Opera Mail/9.63 (Win32)
Xref: news.mathworks.com comp.soft-sys.matlab:525077


On Sun, 15 Mar 2009 13:27:01 -0400, Steffen <rileksn@gmail.com> wrote:

> Hello,
>
> I am kind of confused by the stuff I read the last days about the  
> Distributed/Parallel Computing Toolbox, perhaps someone can give me some  
> general advices.
>
> Very briefly my problem. I have a 300x300 movie and each pixel contains  
> in its third dimension a spectrum which I need to fit. Basically I run  
> for all 90000 pixels two functions, a fitting routine and some math to  
> convert the fitted peaks. Since this takes me about 2 days, I thought of  
> distributing the fitting/conversion routine to other computers (4 to 8  
> PCs).
>
> for i=1:300
>     for j=1:300
>         spectrum=series(i,j,:);
>         peaks=func_fit_spectrum(spectrum);
>         handles=func_peak_conversion(peaks);
>     end
> end
>
> 1) Would that task be solveable via distributed computing or am I  
> missing some major pitfalls here?
> 2) Am I right that the other computers require Matlab and the  
> distributed toolbox to be installed? How about the functions that are  
> called, just copying them to the workers or is the toolbox automatically  
> taking care of that?
> 3) The example given in the help somehow suggests just to replace the  
> for loops by parfor loops. Does that hold true for this particular case  
> with two for loops? Or do I have to cut the main image into small pieces  
> myself and then send it to the workers?
>
> Many thanks in advance for any advice, much appreciated!
>
> Steffen

1) I think parallel computing is a good bet to speed this up. The  
'pitfall' might be the overhead in copying the data to and from workers,  
this would depend on the size of your third dimension, the size of your  
computed output etc.

2)
Yes, you are right. The other computers need to be set up to be 'workers'.  
This is not the same as installing 'MATLAB' since they (workers) can only  
be started/stopped/communicated with by the MATLAB Distributed computing  
engine (what you seem to refer to as the 'distributed toolbox'). You will  
not be able to run a MATLAB session like your desktop on them.

Have a look at:
http://www.mathworks.com/cmsimages/dm_workflow_wl_18819.gif
The 'workers' on the left (Desktop System) refer to process on your  
machine, you just need the PCT for this. If you only wanted to harness  
multiple processors on your desktop, the right side box could be omitted.
The 'workers' on the right refer to process on other machines. There could  
be more than one on a single physical machine depending on how you set it  
up. This setup requires the Distributed server license (in addition to the  
PCT on your desktop) and product to be set up.

More:
http://www.mathworks.com/products/parallel-computing/

3) Nested PARFOR are not supported. You could easily convert your nested  
FOR's to a single for.

air code:

parfor ind=1:300*300
         [i,j]=ind2sub([300 300],ind);     %look up doc ind2sub to get more  
info
         spectrum=series(i,j,:);
         peaks=func_fit_spectrum(spectrum); %arent you overwriting these  
variables?
         handles=func_peak_conversion(peaks);
end