Using the best optimizer for a simulation

Hi, I need help deciding on which optimizer may be best for my purposes. I'm running a simulation that depends on 30 variables (I've been optimizing to 9 just in the interest of time) and is being fit to a data set from a paper. There are no constraints and I've been using patternsearch in order to find the most suitable variables, am I on the right track?

Answer 1

[Walter Roberson]

Probably not. You are doing nonlinear least squares. The algorithms used for fit() from the curvefitting toolbox are pretty efficient if you can happen to get within the basin of attraction. Sometimes the most efficient approach is to run fit() multiple times with different InitialPoint values in the options structure.

[Cpan]

Rather than fitting a curve to the data I generate from the simulation, my simulation (contains reactions and heat transfer components solved through numerical methods) is being fed variables and spitting out some solution, to which I compare to test data. I don't know if I'm misinterpreting what fit() does, but I don't think it achieves my goal.

[Walter Roberson]

Is the result of the comparison to the test data being used to inform which parameters to change and by how much, in order to try to get the closest match to the test data? And is this happening for a number of data points, that you are producing modeled values for a number of inputs, each combination producing one output that is being compared to the corresponding data set from the paper? Are you creating a measure of goodness of fit according to

sum( (modeled - known)^2 )

-- that is trying to minimize the "residue" of the difference between the modeled and the known data?

If these things are true then you are doing a non-linear curve fitting and fit() can be an appropriate routine to try.

The examples for fit() mostly deal with models given by strings such as 'a*exp(b*x)+c' but fittype() accepts a function handle for a custom equation.

[Walter Roberson]

fit() can have difficulty with functions that have the equivalent of a large basin of attraction and a much narrower basin of attraction. fit() does well at finding a "pretty good" area, but it is not a global minimizer.

fminsearch() is better about finding its way around some obstacles, but it is not really a global minimizer either -- and even if you do get to within a good catch basin, fminsearch() probably will not find the local minima (i.e., run an fmincon() call after you are satisfied with the fminsearch() results.)

If you have a basin of attraction with walls and outside the function slopes down asymptotically, then fminsearch() can end up chasing that asymptope for as long as you let it. It knows it is following a downward descent, but has no way of observing that the curve is flattening. And besides, your function just might have a change in slope somewhere past 10^200. There is no "black-box" global minimizer that can guess that beyond

if x < 1; y = -1; else y = 1/x; end

that the function might really be

if x < 1; y = -1; elseif x > 10^exp(pi) & x < 10.0000001^exp(pi); y = 1/x-2; else y = 1/x; end

patternsearch() tends to be slow, and if there is an obvious long slope then it will probably take a long time to cross it. I rarely find it useful to use patternsearch().

See also my discussion at https://www.mathworks.com/matlabcentral/answers/287850-improving-nelder-mead-optimization-by-genetic-algorithms-and-simulated-annealing-concepts#answer_224569 about patternsearch() and simulannealbnd()

[Cpan]

Thank you for the information, I think I'll be able to make a decision from here!

[Walter Roberson]

simulannealbnd is sometimes said to be the only global minimizer that is guaranteed to find the global minimum... that if you let it run long enough, that statistically it will explore the entire space. I have to wonder, though, whether the people who say that have ever looked at the analysis of random walk probability of returning to the origin; http://mathworld.wolfram.com/PolyasRandomWalkConstants.html

... Anyhow.

In my experience you pretty much need a hybrid strategy, with a gross level optimization to (hopefully) get you into the right area, and then a more detailed optimization to get to rock bottom in that area.

Nonlinear least squared fits are pretty lumpy in the fine details, and it is common for the determination of the exact minima position to end up coming down to accidents of round-off due to the order of evaluation.

And see https://www.mathworks.com/matlabcentral/answers/405549-custom-equation-in-curve-fitting#comment_578562