Why does the "fsolve" function in Optimization Toolbox not converge when using a quadratic cost function?

I am using the "fsolve" function to optimize a vector of complex coefficients using a quadratic cost function.

 

I am able to obtain accurate results if I use a linear cost function. However, when I use the quadratic cost function, the solver does not converge. Why does using linear cost converge but using quadratic cost does not converge?

Answer 1

[MathWorks Support Team]

The algorithm in "fsolve" already solves for the quadratic form that you want. See the Algorithm notes in the following documentation page:

https://www.mathworks.com/help/releases/R2022a/optim/ug/fsolve.html#:~:text=%27cg%27.-,Algorithms,-The%20Levenberg%2DMarquardt

 

The following reference page explains more about the algorithm "fsolve" uses:

https://www.mathworks.com/help/releases/R2022a/optim/ug/least-squares-model-fitting-algorithms.html#:~:text=Least%20Squares.-,Levenberg%2DMarquardt%20Method,-The%20least%2Dsquares

If you would still like to solve using a quadratic cost function, you could use the "quadprog" function. 

Another reason that problems do not converge could be scaling. If the initial point is very large when squaring the objective function, this value is also squared. Thus, the solver starts from a much "worse" point.

One possible remedy could be to scale the problem so that the objective values are in a more reasonable range. You might find the "center and scale your problem" section in this documentation page helpful:

https://www.mathworks.com/help/optim/ug/when-the-solver-fails.html#br53duy-1

Answer 2

[Matt J]

Very often, quadratic functions have no roots to find, e.g.,

fsolve(@(x) abs(x).^2+1,  [1+1i;0+3i])

No solution found.

fsolve stopped because the problem appears regular as measured by the gradient,
but the vector of function values is not near zero as measured by the
value of the function tolerance.

ans = 
   0.0000 + 1.0000i
  -0.0000 + 3.0000i