finite difference step size value in fmincon

Hi

I'm using fmincon to solve an optimization problem.

I have question regarding step size in fmincon. When I use finite difference step size (ds) of the magnitude 1e-03, design variable (x) changes from x to x+ds.

But when I use step size (ds) of the magnitude 1, design varible(x) changes from x to 2x or when ds = 2, design variable changes from x to 3x.

Because of this very large change in design variables, the solver exceeds the upper bound (ub) and the solver tries to adjust the bounds which leads to errors.

Why is this happening?

TIA...

[Mario Malic]

From documentation of fmincon - FiniteDifferenceStepSize

delta = v.*sign′(x).*max(abs(x),TypicalX);

As you can see, if your v equals to 1, steps will be equal to your variable.

I am not sure if it is only the delta that determines what x will be in next iteration, so it would be great if someone could verify this.

[Hainan Wang]

Hi, Mario

I am curious for the equation in the document:

delta = v.*sign′(x).*max(abs(x),TypicalX);

What does v, x, and TypicalX mean?

Please enlight me, thank you!

[Mario Malic]

v is the actual FiniteDifferenceStepSize scalar, or vector, if you would like to set the step size for each of your variables.

x is the value of your optimization variable(s) at current iteration

TypicalX is described well in the documentation.

Answer 1

[Matt J]

You might try the sqp or interior point algorithm, instead of whatever fmincon algorithm setting you are currently using. According to the documentation, the sqp algorithm in particular will take finite differencing steps that respect bounds

https://www.mathworks.com/help/optim/ug/constrained-nonlinear-optimization-algorithms.html#bsgppl4

[Akshay Bhapkar]

Hello,

I'm using sqp algorithm.

And sometimes, I get the following warning:

Warning: Derivative finite-differencing step was artificially reduced to be within bound constraints. This may adversely affect convergence. Increasing distance between bound constraints, in dimension 10, to be at least 10.64 may improve results.

I don't know if this warning is related to my earlier query, but I never got such a warning when I worked on a problem where I used step size of the magnitude 1e-03 or 1e-04.

[Matt J]

It sounds like your interval size ub(10)-lb(10) on the 10th parameter is smaller than the specified step-size. Therefore, to keep the steps in bounds, fmincon had to shrink the step-size you had specified.

It's very hard to give advice without seeing the code or a mathematical description of the problem. Is it possible that the 10th parameter is expressed in unsually large units? If you're designing a house, the building dimensions should be expressed in meters, not kilometers... Also, have you considered just providing your own derivative calculation?

[Akshay Bhapkar]

You are right. During the first iteration, with the step size of magnitude 2, the 10th design variable exceeded the ub. But that is because the design variable (x) changes from x to 3x (with step size of 2) which is way too large. I want it to change from x to x+step size.

My question is, why does it (x) not change from x to x + step-size, like when we use step size of magnitude 1e-03 or 1e-04?

Thanks...

[Matt J]

Because the formula used for the actual step is what Mario posted previously

delta = v.*sign′(x).*max(abs(x),TypicalX);