# Compare `lsqnonlin` and `fmincon` for Constrained Nonlinear Least Squares

This example shows that `lsqnonlin` generally takes fewer function evaluations than `fmincon` when solving constrained least-squares problems. Both solvers use the `fmincon` `'interior-point'` algorithm for solving the problem. Yet `lsqnonlin` typically solves problems in fewer function evaluations. The reason is that `lsqnonlin` has more information to work with. `fmincon` minimizes the sum of squares given as $\sum _{i}{F}_{i}^{2}$, where $F$ is a vector function. In contrast, `lsqnonlin` works with the entire vector $F$, meaning it has access to all the components of the sum. In other words, `fmincon` can access only the value of the sum, but `lsqnonlin` can access each component separately.

The `runlsqfmincon` helper function listed at the end of this example creates a series of scaled Rosenbrock-type problems with nonlinear constraints for $N$ ranging from 1 to 50, where the number of problem variables is $2N$. For a description of the Rosenbrock function, see Solve a Constrained Nonlinear Problem, Problem-Based. The function also plots the results, showing:

• Number of iterations

• Number of function counts

• Resulting residuals

The plots show the differences between finite-difference derivative estimates (labeled FD) and derivatives calculated using automatic differentiation. For a description of automatic differentiation, see Automatic Differentiation Background.

`runlsqfmincon;`   The plots show the following results, which are typical.

• For each $N$, the number of iterations for `fmincon` is more than double that of `lsqnonlin` and increases approximately linearly with $N$.

• The number of iterations does not depend on the derivative estimation scheme.

• The function count for finite difference (FD) estimation is much higher than for automatic differentiation.

• The function count for `lsqnonlin` is lower than that for `fmincon` for the same derivative estimation scheme.

• The residuals match for all solution methods, meaning the results are independent of the solver and derivative estimation scheme.

The results indicate that `lsqnonlin` is more efficient than `fmincon` in terms of both iterations and function counts. However, different problems can have different results, and for some problems `fmincon` is more efficient than `lsqnonlin`.

### Helper Function

This code creates the `runlsqfmincon` helper function.

```function [lsq,lsqfd,fmin,fminfd] = runlsqfmincon() optslsq = optimoptions("lsqnonlin",Display="none",... MaxFunctionEvaluations=1e5,MaxIterations=1e4); % Allow for many iterations and Fevals optsfmincon = optimoptions("fmincon",Display="none",... MaxFunctionEvaluations=1e5,MaxIterations=1e4); % Create structures to hold results z = zeros(1,50); lsq = struct('Iterations',z,'Fcount',z,'Residual',z); lsqfd = lsq; fmin = lsq; fminfd = lsq; rng(1) % Reproducible initial points x00 = -1/2 + randn(50,1); y00 = 1/2 + randn(50,1); for N = 1:50 x = optimvar("x",N,LowerBound=-3,UpperBound=3); y = optimvar("y",N,LowerBound=0,UpperBound=9); prob = optimproblem("Objective",sum((10*(y - x.^2)).^2 + (1 - x).^2)); x0.x = x00(1:N); x0.y = y00(1:N); % Include a set of nonlinear inequality constraints cons = optimconstr(N); for i = 1:N cons(i) = x(i)^2 + y(i)^2 <= 1/2 + 1/8*i; end prob.Constraints = cons; [sol,fval,exitflag,output] = solve(prob,x0,Options=optslsq); lsq.Iterations(N) = output.iterations; lsq.Fcount(N) = output.funcCount; lsq.Residual(N) = fval; [sol,fval,exitflag,output] = solve(prob,x0,Options=optslsq,... ObjectiveDerivative='finite-differences',ConstraintDerivative='finite-differences'); lsqfd.Iterations(N) = output.iterations; lsqfd.Fcount(N) = output.funcCount; lsqfd.Residual(N) = fval; [sol,fval,exitflag,output] = solve(prob,x0,Options=optsfmincon,Solver="fmincon"); fmin.Iterations(N) = output.iterations; fmin.Fcount(N) = output.funcCount; fmin.Residual(N) = fval; [sol,fval,exitflag,output] = solve(prob,x0,Options=optsfmincon,Solver="fmincon",... ObjectiveDerivative='finite-differences',ConstraintDerivative='finite-differences'); fminfd.Iterations(N) = output.iterations; fminfd.Fcount(N) = output.funcCount; fminfd.Residual(N) = fval; end N = 1:50; plot(N,lsq.Iterations,'k',N,lsqfd.Iterations,'b--',N,fmin.Iterations,'g',N,fminfd.Iterations,'r--') legend('lsqnonlin','lsqnonlin FD','fmincon','fmincon FD','Location','northwest') xlabel('N') ylabel('Iterations') title('Iterations') figure semilogy(N,lsq.Fcount,'k',N,lsqfd.Fcount,'b--',N,fmin.Fcount,'g',N,fminfd.Fcount,'r--') legend('lsqnonlin','lsqnonlin FD','fmincon','fmincon FD','Location','northwest') xlabel('N') ylabel('log(Function count)') title('Function count, log-scaled') figure plot(N,lsq.Residual,'k',N,lsqfd.Residual,'b--',N,fmin.Residual,'g',N,fminfd.Residual,'r--') legend('lsqnonlin','lsqnonlin FD','fmincon','fmincon FD','Location','southeast') xlabel('N') ylabel('Residual') ylim([0,0.4]) title('Residual') end```