Introduction to Parallel Solutions

On this page…
Interactively Run a Loop in Parallel Run a Batch Job Run a Batch Parallel Loop Run Script as Batch Jobs from the Current Folder Browser Distributing Arrays and Running SPMD

Interactively Run a Loop in Parallel

This section shows how to modify a simple for-loop so that it runs in parallel. This loop does not have a lot of iterations, and it does not take long to execute, but you can apply the principles to larger loops. For these simple examples, you might not notice an increase in execution speed.

1. Suppose your code includes a loop to create a sine wave and plot the waveform:

   for i=1:1024
     A(i) = sin(i*2*pi/1024);
   end
   plot(A)

2. To interactively run code that contains a parallel loop, you first open a MATLAB pool. This reserves a collection of MATLAB worker sessions to run your loop iterations. The MATLAB pool can consist of MATLAB sessions running on your local machine or on a remote cluster:

   matlabpool open local 3

3. With the MATLAB pool reserved, you can modify your code to run your loop in parallel by using a parfor statement:

   parfor i=1:1024
     A(i) = sin(i*2*pi/1024);
   end
   plot(A)

   The only difference in this loop is the keyword parfor instead of for. After the loop runs, the results look the same as those generated from the previous for-loop.

   

   Because the iterations run in parallel in other MATLAB sessions, each iteration must be completely independent of all other iterations. The worker calculating the value for A(100) might not be the same worker calculating A(500). There is no guarantee of sequence, so A(900) might be calculated before A(400). (The MATLAB Editor can help identify some problems with parfor code that might not contain independent iterations.) The only place where the values of all the elements of the array A are available is in the MATLAB client, after the data returns from the MATLAB workers and the loop completes.

4. When you are finished with your code, close the MATLAB pool and release the workers:

   matlabpool close

For more information on parfor-loops, see Parallel for-Loops (parfor).

The examples in this section run on three local workers. With cluster profiles, you can control how many workers run your loops, and whether the workers are local or on a cluster. For more information on profiles, see Cluster Profiles.

You can run Simulink models in parallel loop iterations with the sim command inside your loop. For more information and examples of using Simulink with parfor, see Running Parallel Simulations in the Simulink documentation.

Back to Top

Run a Batch Job

To offload work from your MATLAB session to another session, you can use the batch command. This example uses the for-loop from the last section inside a script.

1. To create the script, type:

   edit mywave

2. In the MATLAB Editor, enter the text of the for-loop:

   for i=1:1024
     A(i) = sin(i*2*pi/1024);
   end

3. Save the file and close the Editor.

4. Use the batch command in the MATLAB Command Window to run your script on a separate MATLAB worker:

   job = batch('mywave')

   

5. The batch command does not block MATLAB, so you must wait for the job to finish before you can retrieve and view its results:

   wait(job)

6. The load command transfers variables from the workspace of the worker to the workspace of the client, where you can view the results:

   load(job, 'A')
   plot(A)

7. When the job is complete, permanently remove its data:

   destroy(job)

Back to Top

Run a Batch Parallel Loop

You can combine the abilities to offload a job and run a parallel loop. In the previous two examples, you modified a for-loop to make a parfor-loop, and you submitted a script with a for-loop as a batch job. This example combines the two to create a batch parfor-loop.

1. Open your script in the MATLAB Editor:

   edit mywave

2. Modify the script so that the for statement is a parfor statement:

   parfor i=1:1024
     A(i) = sin(i*2*pi/1024);
   end

3. Save the file and close the Editor.

4. Run the script in MATLAB with the batch command as before, but indicate that the script should use a MATLAB pool for the parallel loop:

   job = batch('mywave', 'matlabpool', 3)

   This command specifies that three workers (in addition to the one running the batch script) are to evaluate the loop iterations. Therefore, this example uses a total of four local workers, including the one worker running the batch script.

   

5. To view the results:

   wait(job)
   load(job, 'A')
   plot(A)

   The results look the same as before, however, there are two important differences in execution:

   • The work of defining the parfor-loop and accumulating its results are offloaded to another MATLAB session (batch).

   • The loop iterations are distributed from one MATLAB worker to another set of workers running simultaneously (matlabpool and parfor), so the loop might run faster than having only one worker execute it.

6. When the job is complete, permanently remove its data:

   destroy(job)

Back to Top

Run Script as Batch Jobs from the Current Folder Browser

From the Current Folder browser, you can run a MATLAB script as a batch job by browsing to the file's folder, right-clicking the file, and selecting Run Script as Batch Job. The batch job runs on the cluster identified by the current default cluster profile. The following figure shows the menu option to run the script file script1.m:

Running a script as a batch from the browser uses only one worker from the cluster. So even if the script contains a parfor loop or spmd block, it does not open an additional pool of workers on the cluster. These code blocks execute on the single worker used for the batch job. If your batch script requires opening an additional pool of workers, you can run it from the command line, as described in Run a Batch Parallel Loop.

When you run a batch job from the browser, this also opens the Job Monitor. The Job Monitor is a tool that lets you track your job in the scheduler queue. For more information about the Job Monitor and its capabilities, see Job Monitor.

Back to Top

Distributing Arrays and Running SPMD

Distributed Arrays

The workers in a MATLAB pool communicate with each other, so you can distribute an array among the workers. Each worker (or lab) contains part of the array, and all the workers are aware of which portion of the array each worker has.

First, open the MATLAB pool:

matlabpool open  % Use default parallel profile

Use the distributed function to distribute an array among the workers:

M = magic(4) % a 4-by-4 magic square in the client workspace
MM = distributed(M)

Now MM is a distributed array, equivalent to M, and you can manipulate or access its elements in the same way as any other array.

M2 = 2*MM;  % M2 is also distributed, calculation performed on workers
x = M2(1,1) % x on the client is set to first element of M2

When you are finished and have no further need of data from the workers, you can close the MATLAB pool. Data on the workers does not persist from one instance of a MATLAB pool to another.

matlabpool close

Single Program Multiple Data

The single program multiple data (spmd) construct lets you define a block of code that runs in parallel on all the workers (workers) in the MATLAB pool. The spmd block can run on some or all the workers in the pool.

matlabpool    % Use default parallel profile
spmd          % By default uses all workers in the pool
    R = rand(4);
end

This code creates an individual 4-by-4 matrix, R, of random numbers on each worker (or lab) in the pool.

Composites

Following an spmd statement, in the client context, the values from the block are accessible, even though the data is actually stored on the workers. On the client, these variables are called Composite objects. Each element of a composite is a symbol referencing the value (data) on a worker in the pool. Note that because a variable might not be defined on every worker, a Composite might have undefined elements.

Continuing with the example from above, on the client, the Composite R has one element for each worker:

X = R{3};  % Set X to the value of R from worker 3.

The line above retrieves the data from worker 3 to assign the value of X. The following code sends data to worker 3:

X = X + 2;
R{3} = X; % Send the value of X from the client to worker 3.

If the MATLAB pool remains open between spmd statements and the same workers are used, the data on each worker persists from one spmd statement to another.

spmd
    R = R + labindex  % Use values of R from previous spmd.
end

A typical use for spmd is to run the same code on a number of workers, each of which accesses a different set of data. For example:

spmd
    INP = load(['somedatafile' num2str(labindex) '.mat']);
    RES = somefun(INP)
end

Then the values of RES on the workers are accessible from the client as RES{1} from worker 1, RES{2} from worker 2, etc.

There are two forms of indexing a Composite, comparable to indexing a cell array:

• AA{n} returns the values of AA from worker n.

• AA(n) returns a cell array of the content of AA from worker n.

When you are finished with all spmd execution and have no further need of data from the workers, you can close the MATLAB pool.

matlabpool close

Although data persists on the workers from one spmd block to another as long as the MATLAB pool remains open, data does not persist from one instance of a MATLAB pool to another.

For more information about using distributed arrays, spmd, and Composites, see Single Program Multiple Data (spmd).

Back to Top

Key Problems Addressed by Parallel Computing

Determining Product Installation and Versions

© 1984-2012- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS