Learn more about Parallel Computing Toolbox

Using a Supported Scheduler

On this page…
Schedulers and Conditions Coding the Task Function Coding in the Client

Schedulers and Conditions

You can run a parallel job using any type of scheduler. This section illustrates how to program parallel jobs for supported schedulers (job manager, local scheduler, Microsoft Windows HPC Server (including CCS), Platform LSF, PBS Pro, TORQUE, or mpiexec).

To use this supported interface for parallel jobs, the following conditions must apply:

• You must have a shared file system between client and cluster machines

• You must be able to submit jobs directly to the scheduler from the client machine

Note   If all these conditions are not met, you must use the generic scheduler interface with any third-party scheduler running a parallel job, including pmode, matlabpool, spmd, and parfor. See Using the Generic Scheduler Interface.

Back to Top

Coding the Task Function

In this section a simple example illustrates the basic principles of programming a parallel job with a third-party scheduler. In this example, the lab whose labindex value is 1 creates a magic square comprised of a number of rows and columns that is equal to the number of labs running the job (numlabs). In this case, four labs run a parallel job with a 4-by-4 magic square. The first lab broadcasts the matrix with labBroadcast to all the other labs , each of which calculates the sum of one column of the matrix. All of these column sums are combined with the gplus function to calculate the total sum of the elements of the original magic square.

The function for this example is shown below.

function total_sum = colsum
if labindex == 1
    % Send magic square to other labs
    A = labBroadcast(1,magic(numlabs)) 
else
    % Receive broadcast on other labs
    A = labBroadcast(1) 
end

% Calculate sum of column identified by labindex for this lab
column_sum = sum(A(:,labindex))

% Calculate total sum by combining column sum from all labs
total_sum = gplus(column_sum)

This function is saved as the file colsum.m on the path of the MATLAB client. It will be sent to each lab by the job's FileDependencies property.

While this example has one lab create the magic square and broadcast it to the other labs, there are alternative methods of getting data to the labs. Each lab could create the matrix for itself. Alternatively, each lab could read its part of the data from a file on disk, the data could be passed in as an argument to the task function, or the data could be sent in a file contained in the job's FileDependencies property. The solution to choose depends on your network configuration and the nature of the data.

Back to Top

Coding in the Client

As with distributed jobs, you find a scheduler and create a scheduler object in your MATLAB client by using the findResource function. There are slight differences in the arguments for findResource, depending on the scheduler you use, but using configurations to define as many properties as possible minimizes coding differences between the scheduler types.

You can create and configure the scheduler object with this code:

sched = findResource('scheduler', 'configuration', myconfig)

where myconfig is the name of a user-defined configuration for the type of scheduler you are using. Any required differences for various scheduling options are controlled in the configuration. You can have one or more separate configurations for each type of scheduler. For complete details, see Programming with User Configurations. Create or modify configurations according to the instructions of your system administrator.

When your scheduler object is defined, you create the job object with the createParallelJob function.

pjob = createParallelJob(sched);

The function file colsum.m (created in Coding the Task Function) is on the MATLAB client path, but it has to be made available to the labs. One way to do this is with the job's FileDependencies property, which can be set in the configuration you used, or by:

set(pjob, 'FileDependencies', {'colsum.m'})

Here you might also set other properties on the job, for example, setting the number of workers to use. Again, configurations might be useful in your particular situation, especially if most of your jobs require many of the same property settings. To run this example on four labs, you can established this in the configuration, or by the following client code:

set(pjob, 'MaximumNumberOfWorkers', 4)
set(pjob, 'MinimumNumberOfWorkers', 4)

You create the job's one task with the usual createTask function. In this example, the task returns only one argument from each lab, and there are no input arguments to the colsum function.

t = createTask(pjob, @colsum, 1, {})

Use submit to run the job.

submit(pjob)

Make the MATLAB client wait for the job to finish before collecting the results. The results consist of one value from each lab. The gplus function in the task shares data between the labs, so that each lab has the same result.

waitForState(pjob)
results = getAllOutputArguments(pjob)
results = 
    [136]
    [136]
    [136]
    [136]

Back to Top

Introduction

Using the Generic Scheduler Interface

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS