Thread Subject:

Few questions on Distributed Computing Server

Hi all,

I'm in the middle of integrating a project workflow with Matlab's
Distributed Computing Server and HPC 2008. I have a few challenges
that I haven't been able to find answers to in the documentation.

1) Is there a way to send a job to the scheduler that isn't masked in
the Matlab environment. As an example, I want to launch
sampleprogram.exe on a cluster node. When using Matlab's Distributed
Computing Server, the only way I know how to do that is to send the
system() function to it. Then, on the node, an instance of Matlab is
initiated and sampleprogram.exe is executed there. I don't
necessarily need to open up a copy of the Matlab environment to
execute my program - it would be nice to be able to send a job to
execute only sampleprogram.exe.

2) This next difficulty I'm experiencing is that I cannot send more
that four commands out to the cluster. I have four quad-core nodes on
the cluster - each with a Parallel Computing Toolbox license. I would
like to be able to issue four tasks to each of them at a time, but I
get a license error pertaining to the Distributed Computing Server.
Is there a way to have the scheduler issue multiple jobs to a node?

Thanks - I appreciate anyone's input on clarifying these issues.

Stefan,
> 1) Is there a way to send a job to the scheduler that isn't masked in
> the Matlab environment. As an example, I want to launch
> sampleprogram.exe on a cluster node. When using Matlab's Distributed
> Computing Server, the only way I know how to do that is to send the
> system() function to it. Then, on the node, an instance of Matlab is
> initiated and sampleprogram.exe is executed there. I don't
> necessarily need to open up a copy of the Matlab environment to
> execute my program - it would be nice to be able to send a job to
> execute only sampleprogram.exe.

The only way to do this is by using Microsoft's API directly. Using the MATLAB API will always cause a MATLAB worker to start before your .exe is launched. There are various ways of using Microsoft's API (e.g. through MATLAB COM or .NET integration), but probably the simplest is to use Microsoft's command line job submisision. e.g.

>> % This will submit a job to HPC Server 2008 that runs sampleprogram.exe on
>> % one compute node. Note that this uses Microsoft's API directly and not
>> % the API from Parallel Computing Toolbox.
>>commandToRun = 'job submit sampleprogram.exe';
>>system(commandtoRun)

You need to refer to Microsoft's documentation for more details on job submission:
http://technet.microsoft.com/en-us/library/cc972834%28WS.10%29.aspx

> 2) This next difficulty I'm experiencing is that I cannot send more
> that four commands out to the cluster. I have four quad-core nodes on
> the cluster - each with a Parallel Computing Toolbox license. I would
> like to be able to issue four tasks to each of them at a time, but I
> get a license error pertaining to the Distributed Computing Server.
> Is there a way to have the scheduler issue multiple jobs to a node?

Your job can contain as many tasks as you like. How many of these tasks are run at the same time on the cluster nodes is controlled by your scheduler and the number of MATLAB Distributed Computing Server licenses that you have. The fact that you are getting a license error suggests that you do not have enough MATLAB Distributed Computing Server licenses to run more than 4 MATLAB workers. Perhaps someone else is also running MATLAB jobs on the cluster o the licenses are being used elsewhere?

Regards,
Elwin

On Sep 22, 11:29 am, "Elwin Chan" <nos...@nospam.com> wrote:
> Stefan,
>
> > 1)  Is there a way to send a job to the scheduler that isn't masked in
> > the Matlab environment.  As an example, I want to launch
> > sampleprogram.exe on a cluster node.  When using Matlab's Distributed
> > Computing Server, the only way I know how to do that is to send the
> > system() function to it.  Then, on the node, an instance of Matlab is
> > initiated and sampleprogram.exe is executed there.  I don't
> > necessarily need to open up a copy of the Matlab environment to
> > execute my program - it would be nice to be able to send a job to
> > execute only sampleprogram.exe.
>
> The only way to do this is by using Microsoft's API directly.  Using the MATLAB API will always cause a MATLAB worker to start before your .exe is launched.  There are various ways of using Microsoft's API (e.g. through MATLAB COM or .NET integration), but probably the simplest is to use Microsoft's command line job submisision.  e.g.
>
> >> % This will submit a job to HPC Server 2008 that runs sampleprogram.exe on
> >> % one compute node.  Note that this uses Microsoft's API directly and not
> >> % the API from Parallel Computing Toolbox.
> >>commandToRun = 'job submit sampleprogram.exe';
> >>system(commandtoRun)
>
> You need to refer to Microsoft's documentation for more details on job submission:http://technet.microsoft.com/en-us/library/cc972834%28WS.10%29.aspx
>
> > 2) This next difficulty I'm experiencing is that I cannot send more
> > that four commands out to the cluster.  I have four quad-core nodes on
> > the cluster - each with a Parallel Computing Toolbox license.  I would
> > like to be able to issue four tasks to each of them at a time, but I
> > get a license error pertaining to the Distributed Computing Server.
> > Is there a way to have the scheduler issue multiple jobs to a node?
>
> Your job can contain as many tasks as you like.  How many of these tasks are run at the same time on the cluster nodes is controlled by your scheduler and the number of MATLAB Distributed Computing Server licenses that you have.  The fact that you are getting a license error suggests that you do not have enough MATLAB Distributed Computing Server licenses to run more than 4 MATLAB workers.  Perhaps someone else is also running MATLAB jobs on the cluster o the licenses are being used elsewhere?
>
> Regards,
> Elwin

Thanks for your help!

Nice post. Good discussion is going on. Thanks for sharing tis useful information. The post seems to be very infromativ eone. It will be very useful for the readers. Computing is becoming very familiar now a days.

[url=http://cloudjobs.net]Search Cloud Computing Jobs - Free Service for Employers and Candidates[/url]