# Using GOP to Achieve MPI_Allreduce Functionality

In this example, we look at the `gop` function and the functions that build on it: `gplus` and `gcat`. These seemingly simple functions turn out to be very powerful tools in parallel programming.

The `gop` function allows us to perform any associative binary operation on a variable that is defined on all the labs. This allows us not only to sum a variable across all the labs, but also to find its minimum and maximum across the labs, concatenate them, and perform many other useful operations.

Related Documentation:

- spmd reference page in the Parallel Computing Toolbox™ User's Guide

## Contents

The code shown in this example can be found in this function:

```
function paralleltutorial_gop
```

```
%#ok<*NOPRT>: disable code analyzer printing warning
```

## Introduction

When doing parallel programming, we often run into the situation of having a variable defined on all the labs, and we want to perform an operation on the variable as it exists on all the labs. For example, if we enter an spmd statement and define

spmd x = labindex end

Lab 1: x = 1 Lab 2: x = 2 Lab 3: x = 3 Lab 4: x = 4 Lab 5: x = 5 Lab 6: x = 6 Lab 7: x = 7 Lab 8: x = 8 Lab 9: x = 9 Lab 10: x = 10 Lab 11: x = 11 Lab 12: x = 12

on all the labs, we might want to calculate the sum of the values of `x` across the labs. This is exactly what the `gplus` operation does, it sums the `x` across the labs and duplicates the result on all labs:

spmd s = gplus(x); end

The variables assigned to inside an spmd statement are represented on the client as Composite. We can bring the resulting values from the labs to the client by indexing into the Composite much like that of cell arrays:

```
s{1} % Display the value of s on lab 1. All labs store the same value.
```

ans = 78

Also, `gop`, `gplus`, and `gcat` allow us to specify a single lab to which the function output should be returned, and they return an empty vector on the other labs.

spmd s = gplus(x, 1); end s{1}

ans = 78

This example shows how to perform a host of operations similar to addition across all the labs. In MPI, these are known as collective operations, such as MPI_SUM, MPI_PROD, MPI_MIN, MPI_MAX, etc.

## Create the Input Data for Our Examples

The data we use for all our examples is very simple: a 1-by-2 variant array that is only slightly more complicated than the `x` we defined in the beginning:

spmd x = labindex + (1:2) end

Lab 1: x = 2 3 Lab 2: x = 3 4 Lab 3: x = 4 5 Lab 4: x = 5 6 Lab 5: x = 6 7 Lab 6: x = 7 8 Lab 7: x = 8 9 Lab 8: x = 9 10 Lab 9: x = 10 11 Lab 10: x = 11 12 Lab 11: x = 12 13 Lab 12: x = 13 14

## Using GPLUS and GCAT

Now that we have initialized our vector `x` to different values on the labs, we can ask questions such as what is the element-by-element sum of the values of `x` across the labs? What about the product, the minimum, and the maximum? As to be expected from our introduction,

spmd s = gplus(x); end s{1}

ans = 90 102

returns the element-by-element addition of the values of `x`. However, `gplus` is only a special case of the `gop` operation, short for Global OPeration. The `gop` function allows us to perform any associative operation across the labs on the elements of a variant array. The most basic example of an associative operation is addition; it is associative because addition is independent of the grouping which is used:

(a + b) + c = a + (b + c)

In MATLAB®, addition can be denoted by the `@plus` function handle, so we can also write `gplus(x)` as

spmd s = gop(@plus, x); end s{1}

ans = 90 102

We can concatenate the vector `x` across the labs by using the `gcat` function, and we can choose the dimension to concatenate along.

spmd y1 = gcat(x, 1); % Concatenate along rows. y2 = gcat(x, 2); % Concatenate along columns. end y1{1} y2{1}

ans = 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 ans = Columns 1 through 13 2 3 3 4 4 5 5 6 6 7 7 8 8 Columns 14 through 24 9 9 10 10 11 11 12 12 13 13 14

## Other Elementary Uses of GOP

It is simple to calculate the element-by-element product of the values of `x` across the labs:

spmd p = gop(@times, x); end p{1}

ans = 1.0e+10 * 0.6227 4.3589

We can also find the element-by-element maximum of `x` across the labs:

spmd M = gop(@max, x); m = gop(@min, x); end M{1} m{1}

ans = 13 14 ans = 2 3

## Logical Operations

MATLAB has even more built-in associative operations. The logical AND, OR, and XOR operations are represented by the `@and`, `@or`, and `@xor` function handles. For example, look at the logical array

spmd y = (x > 4) end

Lab 1: y = 0 0 Lab 2: y = 0 0 Lab 3: y = 0 1 Lab 4: y = 1 1 Lab 5: y = 1 1 Lab 6: y = 1 1 Lab 7: y = 1 1 Lab 8: y = 1 1 Lab 9: y = 1 1 Lab 10: y = 1 1 Lab 11: y = 1 1 Lab 12: y = 1 1

We can then easily perform these logical operations on the elements of `y` across the labs:

spmd yand = gop(@and, y); yor = gop(@or, y); yxor = gop(@xor, y); end yand{1} yor{1} yxor{1}

ans = 0 0 ans = 1 1 ans = 1 0

## Bitwise Operations

To conclude our tour of the associative operations that are built into MATLAB, we look at the bitwise AND, OR, and XOR operations. These are represented by the `@bitand`, `@bitor`, and `@bitxor` function handles.

spmd xbitand = gop(@bitand, x); xbitor = gop(@bitor, x); xbitxor = gop(@bitxor, x); end xbitand{1} xbitor{1} xbitxor{1}

ans = 0 0 ans = 15 15 ans = 0 12

## Finding Locations of Min and Max

We need to do just a little bit of programming to find the labindex corresponding to where the element-by-element maximum of `x` across the labs occurs. We can do this in just a few lines of code:

```
type pctdemo_aux_gop_maxloc
```

function [val, loc] = pctdemo_aux_gop_maxloc(inval) %PCTDEMO_AUX_GOP_MAXLOC Find maximum value of a variant and its labindex. % [val, loc] = pctdemo_aux_gop_maxloc(inval) returns to val the maximum value % of inval across all the labs. The labindex where this maximum value % resides is returned to loc. % Copyright 2007 The MathWorks, Inc. out = gop(@iMaxLoc, {inval, labindex*ones(size(inval))}); val = out{1}; loc = out{2}; end function out = iMaxLoc(in1, in2) % Calculate the max values and their locations. Return them as a cell array. in1Largest = (in1{1} >= in2{1}); maxVal = in1{1}; maxVal(~in1Largest) = in2{1}(~in1Largest); maxLoc = in1{2}; maxLoc(~in1Largest) = in2{2}(~in1Largest); out = {maxVal, maxLoc}; end

and when the function has been implemented, it can be applied just as easily as any of the built-in operations:

spmd [maxval, maxloc] = pctdemo_aux_gop_maxloc(x); end [maxval{1}, maxloc{1}]

ans = 13 14 12 12

Similarly, we only need a few lines of code to find the labindex where the element-by-element minimum of `x` across the labs occurs:

```
type pctdemo_aux_gop_minloc
```

function [val, loc] = pctdemo_aux_gop_minloc(inval) %PCTDEMO_AUX_GOP_MINLOC Find minimum value of a variant and its labindex. % [val, loc] = pctdemo_aux_gop_minloc(inval) returns to val the minimum value % of inval across all the labs. The labindex where this minimum value % resides is returned to loc. % Copyright 2007 The MathWorks, Inc. out = gop(@iMinLoc, {inval, labindex*ones(size(inval))}); val = out{1}; loc = out{2}; end function out = iMinLoc(in1, in2) % Calculate the min values and their locations. Return them as a cell array. in1Smallest = (in1{1} < in2{1}); minVal = in1{1}; minVal(~in1Smallest) = in2{1}(~in1Smallest); minLoc = in1{2}; minLoc(~in1Smallest) = in2{2}(~in1Smallest); out = {minVal, minLoc}; end

We can then easily find the minimum with `gop`:

spmd [minval, minloc] = pctdemo_aux_gop_minloc(x); end [minval{1}, minloc{1}]

ans = 2 3 1 1