Parallel Computing Toolbox™

Introduction to Parallel Solutions

Interactively Running a Loop in Parallel

This section shows how to modify a simple for-loop so that it runs in parallel. This loop does not have a lot of iterations, and it does not take long to execute, but you can apply the principles to larger loops. For these simple examples, you might not notice an increase in execution speed.

1. Suppose your code includes a loop to create a sine wave and plot the waveform:

   for i=1:1024
     A(i) = sin(i*2*pi/1024);
   end
   plot(A)

2. To interactively run code that contains a parallel loop, you first open a MATLAB pool. This reserves a collection of MATLAB worker sessions to run your loop iterations. The MATLAB pool can consist of MATLAB sessions running on your local machine or on a remote cluster:

   matlabpool open

3. With the MATLAB pool reserved, you can modify your code to run your loop in parallel by using a parfor statement:

   parfor i=1:1024
     A(i) = sin(i*2*pi/1024);
   end
   plot(A)

   The only difference in this loop is the keyword parfor instead of for. After the loop runs, the results look the same as those generated from the previous for-loop.

   

   Because the iterations run in parallel in other MATLAB sessions, each iteration must be completely independent of all other iterations. The worker calculating the value for A(100) might not be the same worker calculating A(500). There is no guarantee of sequence, so A(900) might be calculated before A(400). (The MATLAB Editor can help identify some problems with parfor code that might not contain independent iterations.) The only place where the values of all the elements of the array A are available is in the MATLAB client, after the data returns from the MATLAB workers and the loop completes.

4. When you are finished with your code, close the MATLAB pool and release the workers:

   matlabpool close

For more information on parfor-loops, see Parallel for-Loops (parfor).

The examples in this section run on four local workers, which is the default location and pool size. With parallel configurations, you can control how many workers run your loops, and whether the workers are local or remote. For more information on parallel configurations, see Programming with User Configurations.

Back to Top

Running a Batch Job

To offload work from your MATLAB session to another session, you can use the batch command. This example uses the for-loop from the last section inside an M-file script.

1. To create the script, type:

   edit mywave

2. In the MATLAB Editor, enter the text of the for-loop:

   for i=1:1024
     A(i) = sin(i*2*pi/1024);
   end

3. Save the file and close the Editor.

4. Use the batch command in the MATLAB Command Window to run your script on a separate MATLAB worker:

   job = batch('mywave')

   

5. The batch command does not block MATLAB, so you must wait for the job to finish before you can retrieve and view its results:

   wait(job)

6. The load command transfers variables from the workspace of the worker to the workspace of the client, where you can view the results:

   load(job, 'A')
   plot(A)

7. When the job is complete, permanently remove its data:

   destroy(job)

Back to Top

Running a Batch Parallel Loop

You can combine the abilities to offload a job and run a parallel loop. In the previous two examples, you modified a for-loop to make a parfor-loop, and you submitted a script with a for-loop as a batch job. This example combines the two to create a batch parfor-loop.

1. Open your script in the MATLAB Editor:

   edit mywave

2. Modify the script so that the for statement is a parfor statement:

   parfor i=1:1024
     A(i) = sin(i*2*pi/1024);
   end

3. Save the file and close the Editor.

4. Run the script in MATLAB with the batch command as before, but indicate that the script should use a MATLAB pool for the parallel loop:

   job = batch('mywave', 'matlabpool', 3)

   This command specifies that three workers are to evaluate the loop iterations. When running local workers, the maximum allowed is four workers; one of these is running the batch script, so only three remain for the loop.

   

5. To view the results:

   wait(job)
   load(job, 'A')
   plot(A)

   The results look the same as before, however, there are two important differences in execution:

   • The work of defining the parfor-loop and accumulating its results are offloaded to another MATLAB session (batch).

   • The loop iterations are distributed from one MATLAB worker to another set of workers running simultaneously (matlabpool and parfor), so the loop might run faster than having only one worker execute it.

6. When the job is complete, permanently remove its data:

   destroy(job)

Back to Top

Distributing Arrays in pmode

An array can be partitioned among several MATLAB workers (labs), so that one MATLAB session does not have to accommodate the entire array in memory or perform calculations on every element. These codistributed arrays takes advantage of a cluster's parallel computing and memory resources.

You can work interactively with codistributed arrays in pmode. To get started with pmode and codistributed arrays, see Getting Started with pmode.

You can use parallel jobs for noninteractive (programmatic) solutions involving codistributed arrays. For an introduction to programming jobs to run on a cluster, start with Programming Overview.

Back to Top

Using SPMD and Composites

The single program multiple data (spmd) construct lets you define a block of code that runs in parallel on all the labs (workers) in the MATLAB pool. The spmd block can run on some or all the labs in the pool.

matlabpool    % Use default parallel configuration
spmd          % By default uses all labs in the pool
    R = rand(4);
end

This code creates a 4-by-4 matrix of random numbers on each lab in the pool.

Following an spmd statement, in the client context, the values from the block are accessible, even though the data is actually stored on the labs. On the client, these variables are called Composite objects. Each element of a composite is a symbol referencing the value (data) on a lab in the pool. Note that because a variable might not be defined on every lab, a Composite might have undefined elements.

On the client, the Composite R has one element for each lab:

X = R{3};  % Set X to the value of R from lab 3.

The line above retrieves the data from lab 3 to assign the value of X. The following code sends data to lab 3:

X = X + 2;
R{3} = X; % Send the value of X from the client to lab 3.

If the MATLAB pool remains open between spmd statements and the same labs are used, the data on each lab persists from one spmd statement to another.

spmd
    R = R + labindex  % Use values of R from previous spmd block.
end

A typical use case for spmd blocks is to run the same code on a number of labs, each of which accesses a different set of data. For example:

spmd
    INP = load(['somedatafile' num2str(labindex) '.mat']);
    RES = somefun(INP)
end

Then the values of RES on the labs are accessible from the client as RES{1} from lab 1, RES{2} from lab 2, etc.

There are two forms of indexing a Composite, comparable to indexing a cell array:

• AA{n} returns the values of AA from lab n.

• AA(n) returns a cell array of the content of AA from lab n.

When you are finished with all spmd execution and have no further need of data from the labs, you can close the MATLAB pool.

matlabpool close

Although data persists on the labs from one spmd block to another, it does not persist from one instance of a MATLAB pool to another.

For more information about using spmd and Composites, see Single Program Multiple Data (spmd).

Computing with Interlab Communications

The labs executing an spmd block can communicate with each other during code execution. Therefore, within an spmd block, you can create codistributed arrays using a codistributor object. You can use a for-loop over a distributed range and all of the math functionality enhanced for working with codistributed arrays. For more information about using codistributed arrays, see Math with Codistributed Arrays.

Functions such as labSend, labReceive, and labSendReceive can directly transfer data between the labs. For more information about these and related functions, see Interlab Communication Within a Parallel Job.

Back to Top

Typical Use Cases

Determining Product Installation and Versions