Running on AWS server does not benefit from more cores in parallel pool?

1 view (last 30 days)
I am using the built in Genetic Algorithm functionality in the Optimization Toolbox and time is a serious constraint when running the code.
The problem in question is an integer (binary) problem with linear constraints.
The solutions are on the order of a 400x1 vector of ones and zeroes, and the population size is 500. All other (relevant) Genetic Algorithm options are left as default.
The main bottleneck on run time is within the fitness function (it does not receive the population in vectorized form because this conflicts with using a parallel pool). It has to perform numerous operations on a 3D matrix that is roughly 300000x4x10. These operations are vectorized as much as possible to take advantage of Matlab's speed.
I start a parallel processing pool before calling the algorithm as follows:
c = parcluster('local'); % build the 'local' cluster object
numberOfCores = 6;
With 6 cores it takes roughly five minutes for each generation to complete (when running as a compiled DLL called by .NET). I have tried to set up an AWS server with 20 cores in hopes of speeding up the run time of each generation, but instead the run time per generation was roughly the same or slightly worse. (Matlab verified that the 20 core pool was properly created).
Does anyone know why that would be the case? Is there some way to estimate if there is a "sweet spot" for number of cores in a parallel processing pool compared to any potential overhead associated with parallelizing operations?
  1 Comment
Run Zhu
Run Zhu on 9 Nov 2017
Maybe it is off the topic. But the performance one would obtain is typically: optimization model > optimization algorithm > parallel computing. As in your case, any particular reason to choose GA? Since it is usually slower than intlinprog and patternsearch, consider other solver would be more beneficial.

Sign in to comment.

Answers (2)

Doug Rank
Doug Rank on 9 Nov 2017
I believe I have investigated using other options, but none of them had the necessary components.
The objective function is way too complex to be formulated in a way that would work for intlinprog. (In reality, the solution x for the problem I'm working on only defines a set of rules for how an algorithm within the fitness function will behave to calculate the ultimate value of a given solution).
Patternsearch, if I understand correctly, does not allow for the solution vector to be constrained to integer values, so it would not work in this situation either. If it's particularly susceptible to local minima - that's also a problem.

Walter Roberson
Walter Roberson on 9 Nov 2017
patternsearch is not "particularly susceptible to local minima": it is designed to have a reasonable chance of escaping local minima (but will certainly get stuck if they are deep enough.)
Since all of your inputs are integer (not a mix of integer and non-integer) you can use patternsearch using the techniques described at

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!