Introduction to Parallel Solutions :: Getting Started (Parallel Computing Toolbox™)

Learn more about Parallel Computing Toolbox

Introduction to Parallel Solutions

On this page…
Interactively Running a Loop in Parallel Running a Batch Job Running a Batch Parallel Loop Using Distributed Arrays, spmd, and Composites

On this page…

Interactively Running a Loop in Parallel

Using Distributed Arrays, spmd, and Composites

Interactively Running a Loop in Parallel

This section shows how to modify a simple for-loop so that it runs in parallel. This loop does not have a lot of iterations, and it does not take long to execute, but you can apply the principles to larger loops. For these simple examples, you might not notice an increase in execution speed.

Suppose your code includes a loop to create a sine wave and plot the waveform:
```
for i=1:1024
  A(i) = sin(i*2*pi/1024);
end
plot(A)
```
To interactively run code that contains a parallel loop, you first open a MATLAB pool. This reserves a collection of MATLAB worker sessions to run your loop iterations. The MATLAB pool can consist of MATLAB sessions running on your local machine or on a remote cluster:
```
matlabpool open local 3
```
With the MATLAB pool reserved, you can modify your code to run your loop in parallel by using a parfor statement:
```
parfor i=1:1024
  A(i) = sin(i*2*pi/1024);
end
plot(A)
```
The only difference in this loop is the keyword parfor instead of for. After the loop runs, the results look the same as those generated from the previous for-loop.

Because the iterations run in parallel in other MATLAB sessions, each iteration must be completely independent of all other iterations. The worker calculating the value for A(100) might not be the same worker calculating A(500). There is no guarantee of sequence, so A(900) might be calculated before A(400). (The MATLAB Editor can help identify some problems with parfor code that might not contain independent iterations.) The only place where the values of all the elements of the array A are available is in the MATLAB client, after the data returns from the MATLAB workers and the loop completes.
When you are finished with your code, close the MATLAB pool and release the workers:
```
matlabpool close
```

For more information on parfor-loops, see Parallel for-Loops (parfor).

The examples in this section run on three local workers. With parallel configurations, you can control how many workers run your loops, and whether the workers are local or remote. For more information on parallel configurations, see Programming with User Configurations.

Running a Batch Job

To offload work from your MATLAB session to another session, you can use the batch command. This example uses the for-loop from the last section inside an M-file script.

To create the script, type:
```
edit mywave
```
In the MATLAB Editor, enter the text of the for-loop:
```
for i=1:1024
  A(i) = sin(i*2*pi/1024);
end
```
Save the file and close the Editor.
Use the batch command in the MATLAB Command Window to run your script on a separate MATLAB worker:
```
job = batch('mywave')
```
The batch command does not block MATLAB, so you must wait for the job to finish before you can retrieve and view its results:
```
wait(job)
```
The load command transfers variables from the workspace of the worker to the workspace of the client, where you can view the results:
```
load(job, 'A')
plot(A)
```
When the job is complete, permanently remove its data:
```
destroy(job)
```

Running a Batch Parallel Loop

You can combine the abilities to offload a job and run a parallel loop. In the previous two examples, you modified a for-loop to make a parfor-loop, and you submitted a script with a for-loop as a batch job. This example combines the two to create a batch parfor-loop.

Open your script in the MATLAB Editor:
```
edit mywave
```
Modify the script so that the for statement is a parfor statement:
```
parfor i=1:1024
  A(i) = sin(i*2*pi/1024);
end
```
Save the file and close the Editor.
Run the script in MATLAB with the batch command as before, but indicate that the script should use a MATLAB pool for the parallel loop:
```
job = batch('mywave', 'matlabpool', 3)
```
This command specifies that three workers (in addition to the one running the batch script) are to evaluate the loop iterations. Therefore, this example uses a total of four local workers, including the one worker running the batch script.
To view the results:
```
wait(job)
load(job, 'A')
plot(A)
```
The results look the same as before, however, there are two important differences in execution:
- The work of defining the parfor-loop and accumulating its results are offloaded to another MATLAB session (batch).
- The loop iterations are distributed from one MATLAB worker to another set of workers running simultaneously (matlabpool and parfor), so the loop might run faster than having only one worker execute it.
When the job is complete, permanently remove its data:
```
destroy(job)
```

Using Distributed Arrays, spmd, and Composites

Distributed Arrays

The workers in a MATLAB pool communicate with each other, so you can distribute an array among the labs. Each lab contains part of the array, and all the labs are aware of which portion of the array each lab has.

First, open the MATLAB pool:

matlabpool open  % Use default parallel configuration

Use the distributed function to distribute an array among the labs:

M = magic(4) % a 4-by-4 magic square in the client workspace
MM = distributed(M)

Now MM is a distributed array, equivalent to M, and you can manipulate or access its elements in the same way as any other array.

M2 = 2*MM;  % M2 is also distributed, calculation performed on workers
x = M2(1,1) % x on the client is set to first element of M2

When you are finished and have no further need of data from the labs, you can close the MATLAB pool. Data on the labs does not persist from one instance of a MATLAB pool to another.

matlabpool close

Single Program Multiple Data

The single program multiple data (spmd) construct lets you define a block of code that runs in parallel on all the labs (workers) in the MATLAB pool. The spmd block can run on some or all the labs in the pool.

matlabpool    % Use default parallel configuration
spmd          % By default uses all labs in the pool
    R = rand(4);
end

This code creates an individual 4-by-4 matrix, R, of random numbers on each lab in the pool.

Composites

Following an spmd statement, in the client context, the values from the block are accessible, even though the data is actually stored on the labs. On the client, these variables are called Composite objects. Each element of a composite is a symbol referencing the value (data) on a lab in the pool. Note that because a variable might not be defined on every lab, a Composite might have undefined elements.

Continuing with the example from above, on the client, the Composite R has one element for each lab:

X = R{3};  % Set X to the value of R from lab 3.

The line above retrieves the data from lab 3 to assign the value of X. The following code sends data to lab 3:

X = X + 2;
R{3} = X; % Send the value of X from the client to lab 3.

If the MATLAB pool remains open between spmd statements and the same labs are used, the data on each lab persists from one spmd statement to another.

spmd
    R = R + labindex  % Use values of R from previous spmd.
end

A typical use case for spmd is to run the same code on a number of labs, each of which accesses a different set of data. For example:

spmd
    INP = load(['somedatafile' num2str(labindex) '.mat']);
    RES = somefun(INP)
end

Then the values of RES on the labs are accessible from the client as RES{1} from lab 1, RES{2} from lab 2, etc.

There are two forms of indexing a Composite, comparable to indexing a cell array:

AA{n} returns the values of AA from lab n.
AA(n) returns a cell array of the content of AA from lab n.

When you are finished with all spmd execution and have no further need of data from the labs, you can close the MATLAB pool.

matlabpool close

Although data persists on the labs from one spmd block to another as long as the MATLAB pool remains open, data does not persist from one instance of a MATLAB pool to another.

For more information about using distributed arrays, spmd, and Composites, see Single Program Multiple Data (spmd).

Typical Use Cases

Determining Product Installation and Versions

Recommended Products

Get MATLAB trial software

Includes the most popular MATLAB recorded presentations with Q&A sessions led by MATLAB experts.

Get the Interactive Kit