Thread Subject: Matlabpool configuration Problem

Dear All,

I am a new user, and I apologise if my question has already been answered in previous posts.

I have a problem with the configuration of matlabpool.
I am using matlab on a server that has 32 parallel processors, and I programmed some parfor loops to speed up the computations.

I open matlabpool with the local default configuration, which "connects to 8 labs". Obviously this increases the computational speed by roughly 8 times. But since the server has 32 processors, it seems to me that I could do much better than that. The problem is that I am unable to manually configure matlabpool to manage more parallel labs.

Any suggestion about how to proceed to solve this problem will be greatly appreciated!

Andrea

Hi Andrea,

It looks like you have the parallel processing toolbox, but no license for the distributed computing server.

Thus, you are limited to having a maximum number of 8 workers, which have to be on the machine on which your Matlab session is running.

 

"Jonas" <remove_this_iop@gmx.at> wrote in message <hc4gdh$2b5$1@fred.mathworks.com>...
> Hi Andrea,
>
> It looks like you have the parallel processing toolbox, but no license for the distributed computing server.
>
> Thus, you are limited to having a maximum number of 8 workers, which have to be on the machine on which your Matlab session is running.
>
>
Hi Jonas,

thank you for yor answer. I may be wrong, but it seems to me the software is there. The problem is that I find it difficult to understand how to configure it.
For example, from the "Configurations Manager" I tried to configure a "mpiexec" configuration. When I validate it, it passes the "find resources" test, but it fails the "parallel job" test. I paste below the error reported. In any case, thanks a lot for your help.

best,

Andrea


---------------------------------
'/opt/local/matlab_2009b/unix/bin/glnxa64/mpiexec' -n 10 -env MDCE_DECODE_FUNCTION 'decodeMpiexecSingleTask' -env MDCE_STORAGE_LOCATION 'PC{}:UNIX{/exports/homes/economiques/acaggese/.matlab/local_scheduler_data/R2009b/}:' -env MDCE_STORAGE_CONSTRUCTOR 'makeFileStorageObject' -env MDCE_JOB_LOCATION 'Job1' -env MDCE_WORLD_SIZE '10' 'worker' -parallel
Please specify an authentication passphrase for smpd: op_connect error: socket connection failed, error stack:
MPIDU_Socki_handle_connect(795): connection failure (set=1,sock=16777216,errno=111:Connection refused)
unable to connect mpiexec tree, socket connection failed, error stack:
MPIDU_Socki_handle_connect(795): connection failure (set=1,sock=16777216,errno=111:Connection refused).
  -----------------------------------

Andrea,

What is $MPIEXEC_RSH set to? If it's not set to 'rsh', try setting it first.

That said, is there a reason to use mpiexec? Do you have scheduler installed (LSF, PBS Pro, etc.)? Or could you use the MathWorks Job Manager. Any of these might be easier.

Raymond

"Andrea Caggese" <andrea.caggese@upf.edu> wrote in message <hc6ffi$meb$1@fred.mathworks.com>...
> "Jonas" <remove_this_iop@gmx.at> wrote in message <hc4gdh$2b5$1@fred.mathworks.com>...
> > Hi Andrea,
> >
> > It looks like you have the parallel processing toolbox, but no license for the distributed computing server.
> >
> > Thus, you are limited to having a maximum number of 8 workers, which have to be on the machine on which your Matlab session is running.
> >
> >
> Hi Jonas,
>
> thank you for yor answer. I may be wrong, but it seems to me the software is there. The problem is that I find it difficult to understand how to configure it.
> For example, from the "Configurations Manager" I tried to configure a "mpiexec" configuration. When I validate it, it passes the "find resources" test, but it fails the "parallel job" test. I paste below the error reported. In any case, thanks a lot for your help.
>
> best,
>
> Andrea
>
>
> ---------------------------------
> '/opt/local/matlab_2009b/unix/bin/glnxa64/mpiexec' -n 10 -env MDCE_DECODE_FUNCTION 'decodeMpiexecSingleTask' -env MDCE_STORAGE_LOCATION 'PC{}:UNIX{/exports/homes/economiques/acaggese/.matlab/local_scheduler_data/R2009b/}:' -env MDCE_STORAGE_CONSTRUCTOR 'makeFileStorageObject' -env MDCE_JOB_LOCATION 'Job1' -env MDCE_WORLD_SIZE '10' 'worker' -parallel
> Please specify an authentication passphrase for smpd: op_connect error: socket connection failed, error stack:
> MPIDU_Socki_handle_connect(795): connection failure (set=1,sock=16777216,errno=111:Connection refused)
> unable to connect mpiexec tree, socket connection failed, error stack:
> MPIDU_Socki_handle_connect(795): connection failure (set=1,sock=16777216,errno=111:Connection refused).
> -----------------------------------

Thanks a lot, Raymond, I will try your suggestions.

Andrea



"Raymond Norris" <raymond.norris@mathworks.com> wrote in message <hc7430$cok$1@fred.mathworks.com>...
> Andrea,
>
> What is $MPIEXEC_RSH set to? If it's not set to 'rsh', try setting it first.
>
> That said, is there a reason to use mpiexec? Do you have scheduler installed (LSF, PBS Pro, etc.)? Or could you use the MathWorks Job Manager. Any of these might be easier.
>
> Raymond
>
> "Andrea Caggese" <andrea.caggese@upf.edu> wrote in message <hc6ffi$meb$1@fred.mathworks.com>...
> > "Jonas" <remove_this_iop@gmx.at> wrote in message <hc4gdh$2b5$1@fred.mathworks.com>...
> > > Hi Andrea,
> > >
> > > It looks like you have the parallel processing toolbox, but no license for the distributed computing server.
> > >
> > > Thus, you are limited to having a maximum number of 8 workers, which have to be on the machine on which your Matlab session is running.
> > >
> > >
> > Hi Jonas,
> >
> > thank you for yor answer. I may be wrong, but it seems to me the software is there. The problem is that I find it difficult to understand how to configure it.
> > For example, from the "Configurations Manager" I tried to configure a "mpiexec" configuration. When I validate it, it passes the "find resources" test, but it fails the "parallel job" test. I paste below the error reported. In any case, thanks a lot for your help.
> >
> > best,
> >
> > Andrea
> >
> >
> > ---------------------------------
> > '/opt/local/matlab_2009b/unix/bin/glnxa64/mpiexec' -n 10 -env MDCE_DECODE_FUNCTION 'decodeMpiexecSingleTask' -env MDCE_STORAGE_LOCATION 'PC{}:UNIX{/exports/homes/economiques/acaggese/.matlab/local_scheduler_data/R2009b/}:' -env MDCE_STORAGE_CONSTRUCTOR 'makeFileStorageObject' -env MDCE_JOB_LOCATION 'Job1' -env MDCE_WORLD_SIZE '10' 'worker' -parallel
> > Please specify an authentication passphrase for smpd: op_connect error: socket connection failed, error stack:
> > MPIDU_Socki_handle_connect(795): connection failure (set=1,sock=16777216,errno=111:Connection refused)
> > unable to connect mpiexec tree, socket connection failed, error stack:
> > MPIDU_Socki_handle_connect(795): connection failure (set=1,sock=16777216,errno=111:Connection refused).
> > -----------------------------------