Fmincon cannot move away from initial point

I am trying to run fmincon with 70 design variables. It performs the 0th iteration for the partial derivatives without problems.

Then in iteration 1, it changes values of the variables for 2-3 evaluations, after which fmincon starts decreasing and decreasing delta x with respect to the initial point every evaluation, until delta x gets below the threshold value and the optimisation stops. The result is that basically nothing happened: optimum design vector equals initial design vector.

Additional notes:

- Design vector, constraints and objective value are normalised.

- Some of the design variables are surrogates values. (IDF optimisation is a requirement) During analysis that happens every evaluation, the real values for these variables are computed. Then in the constraints that are passed to fmincon, the surrogates from the design vector and the real values that are computed, are constrained with equality constraints.

Does anyone have experience with such issue, and how to solve it?

Edit: To add to the above, after delta x falls below the threshold, not all constraints are entirely satisfied, even though the design vector is very close to the initial vector. So when it stops, it takes the only feasible design that is available, which is the initial design.

[Matt J]

Then in iteration 1, it changes values of the variables for 2-3 evaluations after which fmincon starts decreasing ....The result is that basically nothing happened: optimum design vector equals initial design vector.

The above two statements seem to contradict each other. If you witness a decrease in the function, how is it also true that "nothing happened"?

Some of the design variables are surrogates values. (IDF optimisation is a requirement)

This needs clarification, like the meaning of "surrogate" and "IDF".

[GForce]

Sorry for the confusion, what I mean is:

- It tries around 20 evaluations in iteration 1 where each evaluation makes a smaller step away from the initial point, but finally the accepted design vector for iteration 1 is the initial design vector, and the accepted value of fval is the initial value of the objective function.

- IDF and surrogate: There are multiple analyses blocks, where each block has as input some design variables, but also some outputs of other analysis blocks. In an IDF scheme, the blocks are not allowed to pass their output to another analysis block. Instead, they use a "random" number, the surrogate value. This is one of the variables of the design vector.

For example:

Suppose there are two analysis code blocks, X and Y. X requires variable a, which is an output of Y. Then fmincon passes an a* to X, and in the constraints sets a* equal to a, which comes out of Y.

Answer 1

[Matt J]

The typical reason why an Optimization Toolbox solver does not move from the initial point is that you have given it a piece-wise constant function, for example because your objective uses discretizing operations like round, ceil, floor, etc... These are illegal first of all because they are non-differentiable, violating smoothness assumptions made by fmincon and friends.

Moreover, though, piecewise constant functions have local minima at all points except at the discontinuous boundaries between the pieces. Therefore, you stand an excellent chance of getting stuck at any starting point you choose.

[Esi Tesi]

For the case that any of the discretizing operators are used, do you have any idea to solve this problem? I mean avoiding to get stuck in local minima, like applying a specific value for any of options in fmincon.

[Matt J]

For the case that any of the discretizing operators are used, do you have any idea to solve this problem?

These operators should not be used. An appropriate alternative to them would depend on the specifics of the objective function, and what it's trying to compute.

[Esi Tesi]

Ok, thank you.

Answer 2

[Walter Roberson]

fmincon() is a local optimizer, and you are trapped in a local minima. The other points that were evaluated at did not satisfy the constraints any better than the original point did.

Note: the non-linear constraint function is not used to constrain values for the first few calls to the objective function, as the first few calls to the objective function are used to estimate the Hessian. The linear constraints are used in determining where to probe the Hessian, if I recall correctly.

Also, in general, the algorithms do not necessarily call the nonlinear constraint function until a feasible location is found and then call the objective function there. They might do that, mostly, but only some of the algorithms promise to satisfy the non-linear constraints at every step.

[GForce]

Thanks for the answer, then I'd like to ask:

About the constraints, I know that every evaluation both objective function and constraint function are called, since I write them to a file. I have now written the objective function to be minimising one of the design variables. So the only thing that stops it is satisfying the constraints.

I have the feeling that this is difficult because an example in my case is: to satisfy a certain constraint c1, 8 of the design variables need to have a certain value each. Then when the design vector is changed, the values that are compared in c1 change as well, and the 8 design variables involved again have to have a certain value each, different from previous time. But the direction in which they need to change, could be the same or could be different than previous time. E.g. x(1) was reduced and constraint c1 got closer to be satisfied. But in the next set of design variables, x(1) should be increased again to make c1 closer to be satisfied. (If it helps, the 8 variables I mention are coefficients for a bezier curve, and the curve they describe influence the values compared in c1)

Would fmincon have trouble in such a case? If so what would be a solution?

Answer 3

[Alan Weiss]

I suggest that you take a look at Final Point Equals Initial Point and also Optimizing a Simulation or Ordinary Differential Equation, especially the part about finite difference step sizes. And maybe consult other parts of the documentation about improving solutions (the other part of the Final Point Equals Initial Point section).

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation