Fminsearchcon

Extension of fminsearchbnd for general inequality and linear inequality constraints

Description

Nonlinear optimization problems with bound constraints can be solved using FMINSEARCHBND (as well as using many other tools.) For the user who has a problem with linear inequality constraints and/or general nonlinear inequalities as well as bound constraints, FMINSEARCHCON provides a toolbox free alternative, based on FMINSEARCH.

FMINSEARCHCON provides a simple, though carefully coded optimization code based on the Matlab provided FMINSEARCH. It uses transformations to deal with the bound constraints, but a penalty function approach to deal with the inequality constraints. The actual objective function is never evaluated outside of the bounds, as the bounds are always tested first.

Note that while FMINSEARCHCON does everything that FMINSEARCHBND is capable of and more, I've chosen to provide both these tools independently because FMINSEARCHCON has a more complex calling sequence. My expectation is that bound constraints are more commonly seen than general inequality constraints.

Acknowledgements

The author wishes to acknowledge the following in the creation of this submission:
fminsearchbnd
This submission has inspired the following:
optimize

• MATLAB release: MATLAB 7.0.1 (R14SP1)
• Published M Files: fminsearchcon_demo
• Other Files: FMINSEARCHCON/demo/fminsearchcon_demo.m, ; FMINSEARCHCON/doc/Understanding_fminsearchbnd.rtf, ; FMINSEARCHCON/doc/Understanding_fminsearchcon.rtf, ; FMINSEARCHCON/fminsearchcon.m, ; license.txt

[abi dema]

 

[Viet Ha Tran]

Dear Mr. John D'Errico,

Thanks for your program. It is quite useful for me. However, I am having a problem:

I try to use fminsearchcom to seek for MLE estimation but although the solution is found as a converged result (exitflag>0), it seems not to be the optimal point. For example, when I pick a parameter and compare the objective function at the estimated point and the two other points at the two side (very close to the estimated one) while keeping the other parameters being the estimated values, the estimated point is not the lowest value. I even try to program so that the program will continue another trial from the converged estimated solution if such issue still exists but it never reach that 'optimal' point. I also check if the result reach the bounds (constant, linear, and nonlinear restrictions) but the solution is quite inside the space.

I wonder why the program cannot reach that point and what I should do to improve it. Since if I use that estimates, I cannot calculate the confidence interval.

Thanks and hope to get your feedback soon,

TV Ha

[John D'Errico]

Please, if you have questions, send them directly to my e-mail address.

The question posed is hard to answer, since I cannot know what has happened, why the code has converged, what constraints are involved, etc. If the final result satisfied the convergence tolerances, then it will terminate the optimization.

If there are only a couple of parameters in the MLE, I'd plot the response surface to get an understanding of what has happened. With 3 parameters, one might look at isosurfaces or streamlines.

John

[Viet Ha Tran]

Thanks for your reply, I have solved this problem. I will post directly to your mail.

TV Ha.

[Haochuan Zhang]

Hi, excellent program. However, I encountered a problem where an optimization problem with bounded constraint and the same problem with euqivalent linear inequility constraint reach different result.

[John D'Errico]

Use of a linear inequality as opposed to a simple bound is different, since the bounds are implemented via transformation, and the inequality constraints are implemented using infinite penalties. So the circumstance where these two classes of constraints yield different result does not surprise me, since when you change the constraint implementation, the transformation changes the shape of the function. This mean that a given starting value may be drawn to a different attractor after transformation.

Using a nonlinear optimizer, there is no assurance of always converging to the globally optimal result. The best we can hope for is merely a local optimum for any method.

John

[Scott Storck]

Code quality and documentation are excellent. Thank you for eliminating my dependency on the Optimization toolbox.