# Minimizing an Expensive Optimization Problem Using Parallel Computing Toolbox™

This example shows how to how to speed up the minimization of an expensive optimization problem using functions in Optimization Toolbox™ and Global Optimization Toolbox. In the first part of the example we solve the optimization problem by evaluating functions in a serial fashion and in the second part of the example we solve the same problem using the parallel for loop (`parfor`) feature by evaluating functions in parallel. We compare the time taken by the optimization function in both cases.

## Contents

## Expensive Optimization Problem

For the purpose of this example, we solve a problem in four variables, where the objective and constraint functions are made artificially expensive by pausing.

function f = expensive_objfun(x) %EXPENSIVE_OBJFUN An expensive objective function used in optimparfor example. % Copyright 2007-2013 The MathWorks, Inc. % Simulate an expensive function by pausing pause(0.1) % Evaluate objective function f = exp(x(1)) * (4*x(3)^2 + 2*x(4)^2 + 4*x(1)*x(2) + 2*x(2) + 1);

function [c,ceq] = expensive_confun(x) %EXPENSIVE_CONFUN An expensive constraint function used in optimparfor example. % Copyright 2007-2013 The MathWorks, Inc. % Simulate an expensive function by pausing pause(0.1); % Evaluate constraints c = [1.5 + x(1)*x(2)*x(3) - x(1) - x(2) - x(4); -x(1)*x(2) + x(4) - 10]; % No nonlinear equality constraints: ceq = [];

## Minimizing Using `fmincon`

We are interested in measuring the time taken by `fmincon` in serial so that we can compare it to the parallel time.

startPoint = [-1 1 1 -1]; options = optimoptions('fmincon','Display','iter','Algorithm','interior-point'); startTime = tic; xsol = fmincon(@expensive_objfun,startPoint,[],[],[],[],[],[],@expensive_confun,options); time_fmincon_sequential = toc(startTime); fprintf('Serial FMINCON optimization takes %g seconds.\n',time_fmincon_sequential);

First-order Norm of Iter F-count f(x) Feasibility optimality step 0 5 1.839397e+00 1.500e+00 3.211e+00 1 11 -9.760099e-01 3.708e+00 7.902e-01 2.362e+00 2 16 -1.480976e+00 0.000e+00 8.344e-01 1.069e+00 3 21 -2.601599e+00 0.000e+00 8.390e-01 1.218e+00 4 29 -2.823630e+00 0.000e+00 2.598e+00 1.118e+00 5 34 -3.905338e+00 0.000e+00 1.210e+00 7.302e-01 6 39 -6.212992e+00 3.934e-01 7.372e-01 2.405e+00 7 44 -5.948762e+00 0.000e+00 1.784e+00 1.905e+00 8 49 -6.940062e+00 1.233e-02 7.668e-01 7.553e-01 9 54 -6.973887e+00 0.000e+00 2.549e-01 3.920e-01 10 59 -7.142993e+00 0.000e+00 1.903e-01 4.735e-01 11 64 -7.155325e+00 0.000e+00 1.365e-01 2.626e-01 12 69 -7.179122e+00 0.000e+00 6.336e-02 9.115e-02 13 74 -7.180116e+00 0.000e+00 1.069e-03 4.670e-02 14 79 -7.180409e+00 0.000e+00 7.799e-04 2.815e-03 15 84 -7.180410e+00 0.000e+00 6.070e-06 3.122e-04 Local minimum found that satisfies the constraints. Optimization completed because the objective function is non-decreasing in feasible directions, to within the default value of the optimality tolerance, and constraints are satisfied to within the default value of the constraint tolerance. Serial FMINCON optimization takes 16.94 seconds.

## Minimizing Using Genetic Algorithm

Since `ga` usually takes many more function evaluations than `fmincon`, we remove the expensive constraint from this problem and perform unconstrained optimization instead. Pass empty matrices `[]` for constraints. In addition, we limit the maximum number of generations to 15 for `ga` so that `ga` can terminate in a reasonable amount of time. We are interested in measuring the time taken by `ga` so that we can compare it to the parallel `ga` evaluation. Note that running `ga` requires Global Optimization Toolbox.

rng default % for reproducibility try gaAvailable = false; nvar = 4; gaoptions = optimoptions('ga','MaxGenerations',15,'Display','iter'); startTime = tic; gasol = ga(@expensive_objfun,nvar,[],[],[],[],[],[],[],gaoptions); time_ga_sequential = toc(startTime); fprintf('Serial GA optimization takes %g seconds.\n',time_ga_sequential); gaAvailable = true; catch ME warning(message('optimdemos:optimparfor:gaNotFound')); end

Best Mean Stall Generation f-count f(x) f(x) Generations 1 100 -5.546e+05 1.483e+15 0 2 150 -5.581e+17 -1.116e+16 0 3 200 -7.556e+17 6.679e+22 0 4 250 -7.556e+17 -7.195e+16 1 5 300 -9.381e+27 -1.876e+26 0 6 350 -9.673e+27 -7.497e+26 0 7 400 -4.511e+36 -9.403e+34 0 8 450 -5.111e+36 -3.011e+35 0 9 500 -7.671e+36 9.346e+37 0 10 550 -1.52e+43 -3.113e+41 0 11 600 -2.273e+45 -4.67e+43 0 12 650 -2.589e+47 -6.281e+45 0 13 700 -2.589e+47 -1.015e+46 1 14 750 -8.149e+47 -5.855e+46 0 15 800 -9.503e+47 -1.29e+47 0 Optimization terminated: maximum number of generations exceeded. Serial GA optimization takes 80.1669 seconds.

## Setting Parallel Computing Toolbox

The finite differencing used by the functions in Optimization Toolbox to approximate derivatives is done in parallel using the `parfor` feature if Parallel Computing Toolbox is available and there is a parallel pool of workers. Similarly, `ga`, `gamultiobj`, and `patternsearch` solvers in Global Optimization Toolbox evaluate functions in parallel. To use the `parfor` feature, we use the `parpool` function to set up the parallel environment. The computer on which this example is published has four cores, so `parpool` starts four MATLAB® workers. If there is already a parallel pool when you run this example, we use that pool; see the documentation for `parpool` for more information.

if max(size(gcp)) == 0 % parallel pool needed parpool % create the parallel pool end

Starting parallel pool (parpool) using the 'local' profile ... connected to 4 workers.

## Minimizing Using Parallel `fmincon`

To minimize our expensive optimization problem using the parallel `fmincon` function, we need to explicitly indicate that our objective and constraint functions can be evaluated in parallel and that we want `fmincon` to use its parallel functionality wherever possible. Currently, finite differencing can be done in parallel. We are interested in measuring the time taken by `fmincon` so that we can compare it to the serial `fmincon` run.

options = optimoptions(options,'UseParallel',true); startTime = tic; xsol = fmincon(@expensive_objfun,startPoint,[],[],[],[],[],[],@expensive_confun,options); time_fmincon_parallel = toc(startTime); fprintf('Parallel FMINCON optimization takes %g seconds.\n',time_fmincon_parallel);

First-order Norm of Iter F-count f(x) Feasibility optimality step 0 5 1.839397e+00 1.500e+00 3.211e+00 1 11 -9.760099e-01 3.708e+00 7.902e-01 2.362e+00 2 16 -1.480976e+00 0.000e+00 8.344e-01 1.069e+00 3 21 -2.601599e+00 0.000e+00 8.390e-01 1.218e+00 4 29 -2.823630e+00 0.000e+00 2.598e+00 1.118e+00 5 34 -3.905338e+00 0.000e+00 1.210e+00 7.302e-01 6 39 -6.212992e+00 3.934e-01 7.372e-01 2.405e+00 7 44 -5.948762e+00 0.000e+00 1.784e+00 1.905e+00 8 49 -6.940062e+00 1.233e-02 7.668e-01 7.553e-01 9 54 -6.973887e+00 0.000e+00 2.549e-01 3.920e-01 10 59 -7.142993e+00 0.000e+00 1.903e-01 4.735e-01 11 64 -7.155325e+00 0.000e+00 1.365e-01 2.626e-01 12 69 -7.179122e+00 0.000e+00 6.336e-02 9.115e-02 13 74 -7.180116e+00 0.000e+00 1.069e-03 4.670e-02 14 79 -7.180409e+00 0.000e+00 7.799e-04 2.815e-03 15 84 -7.180410e+00 0.000e+00 6.070e-06 3.122e-04 Local minimum found that satisfies the constraints. Optimization completed because the objective function is non-decreasing in feasible directions, to within the default value of the optimality tolerance, and constraints are satisfied to within the default value of the constraint tolerance. Parallel FMINCON optimization takes 13.9436 seconds.

## Minimizing Using Parallel Genetic Algorithm

To minimize our expensive optimization problem using the `ga` function, we need to explicitly indicate that our objective function can be evaluated in parallel and that we want `ga` to use its parallel functionality wherever possible. To use the parallel `ga` we also require that the 'Vectorized' option be set to the default (i.e., 'off'). We are again interested in measuring the time taken by `ga` so that we can compare it to the serial `ga` run. Though this run may be different from the serial one because `ga` uses a random number generator, the number of expensive function evaluations is the same in both runs. Note that running `ga` requires Global Optimization Toolbox.

rng default % to get the same evaluations as the previous run if gaAvailable gaoptions = optimoptions(gaoptions,'UseParallel',true); startTime = tic; gasol = ga(@expensive_objfun,nvar,[],[],[],[],[],[],[],gaoptions); time_ga_parallel = toc(startTime); fprintf('Parallel GA optimization takes %g seconds.\n',time_ga_parallel); end

Best Mean Stall Generation f-count f(x) f(x) Generations 1 100 -5.546e+05 1.483e+15 0 2 150 -5.581e+17 -1.116e+16 0 3 200 -7.556e+17 6.679e+22 0 4 250 -7.556e+17 -7.195e+16 1 5 300 -9.381e+27 -1.876e+26 0 6 350 -9.673e+27 -7.497e+26 0 7 400 -4.511e+36 -9.403e+34 0 8 450 -5.111e+36 -3.011e+35 0 9 500 -7.671e+36 9.346e+37 0 10 550 -1.52e+43 -3.113e+41 0 11 600 -2.273e+45 -4.67e+43 0 12 650 -2.589e+47 -6.281e+45 0 13 700 -2.589e+47 -1.015e+46 1 14 750 -8.149e+47 -5.855e+46 0 15 800 -9.503e+47 -1.29e+47 0 Optimization terminated: maximum number of generations exceeded. Parallel GA optimization takes 25.3579 seconds.

## Compare Serial and Parallel Time

X = [time_fmincon_sequential time_fmincon_parallel]; Y = [time_ga_sequential time_ga_parallel]; t = [0 1]; plot(t,X,'r--',t,Y,'k-') ylabel('Time in seconds') legend('fmincon','ga') ax = gca; ax.XTick = [0 1]; ax.XTickLabel = {'Serial' 'Parallel'}; axis([0 1 0 ceil(max([X Y]))]) title('Serial Vs. Parallel Times')

Utilizing parallel function evaluation via `parfor` improved the efficiency of both `fmincon` and `ga`. The improvement is typically better for expensive objective and constraint functions.