Optimization Toolbox

Nonlinear Programming

Optimization Toolbox provides widely used optimization algorithms for solving nonlinear programming problems in MATLAB. The toolbox includes solvers for unconstrained and constrained nonlinear optimization and solvers for least-squares optimization.

Unconstrained Nonlinear Optimization

Optimization Toolbox uses three algorithms to solve unconstrained nonlinear minimization problems:

• The Quasi-Newton algorithm uses a mixed quadratic and cubic line search procedure and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula for updating the approximation of the Hessian matrix.
• The Nelder-Mead algorithm (or downhill simplex) is a direct-search algorithm that uses only function values (does not require derivatives) and handles nonsmooth objective functions. Global Optimization Toolbox provides additional derivative-free optimization algorithms for nonlinear optimization.
• The trust-region algorithm is used for unconstrained nonlinear problems and is especially useful for large-scale problems where sparsity or structure can be exploited.

Unconstrained nonlinear programming used to search an engine performance map for peak efficiency.

Constrained Nonlinear Optimization

Constrained nonlinear optimization problems are composed of nonlinear objective functions and may be subject to linear and nonlinear constraints. Optimization Toolbox uses four algorithms to solve these problems:

• The interior point algorithm is used for general nonlinear optimization. It is especially useful for large-scale problems that have sparsity or structure, and tolerates user-defined objective and constraint function evaluation failures. It is based on a barrier function, and optionally keeps all iterates strictly feasible with respect to bounds during the optimization run.
• The SQP algorithm is used for general nonlinear optimization. It honors bounds at all iterations and tolerates user-defined objective and constraint function evaluation failures.
• The active-set algorithm is used for general nonlinear optimization.
• The trust-region reflective algorithm is used for bound constrained problems or linear equalities only. It is especially useful for large-scale problems.

The interior point and trust-region reflective algorithms enable you to estimate Hessians using different approaches.

For the interior point algorithm, you can estimate Hessians using:

• BFGS (dense)
• Limited memory BFGS (for large-scale problems)
• Hessian-multiply function
• Actual Hessian (sparse or dense)
• Finite difference of gradients, without requiring knowledge of sparsity structure

For the trust-region reflective algorithm, you can use:

• Finite difference of gradients, Hessian with known sparsity structure
• Actual Hessian (sparse or dense)
• Hessian-multiply function

Additionally, the interior point and trust-region reflective algorithms enable you to calculate Hessian-times-vector products in a function without having to form the Hessian matrix explicitly.

Optimization Toolbox also includes an interface to Ziena Optimization's KNITRO^® libraries for solving constrained nonlinear optimization problems.

Constrained nonlinear programming used to design an optimal suspension system.