File Exchange

## LMFsolve.m: Levenberg-Marquardt-Fletcher algorithm for nonlinear least squares problems

version 1.3.0.0 (8.21 KB) by Miroslav Balda

### Miroslav Balda (view profile)

LMFsolve.m finds least-squares solution of an overdetermined system of nonlinear equations

Updated 11 Feb 2009

The function LMFsolve.m serves for finding optimal solution of an overdetermined system of nonlinear equations in the least-squares sense. The standard Levenberg- Marquardt algorithm was modified by Fletcher and coded in FORTRAN many years ago. LMFsolve is its essentially shortened version implemented in MATLAB and complemented by setting iteration parameters as options. This part of the code has been strongly influenced by Duane Hanselman's function mmfsolve.m. Next to it, a finite difference approximation of Jacobian matrix is appended to it as a nested subfunction as well as a function for dispaying of intermediate results.

Calling of the function is rather simple:
[x,ssq,cnt] = LMFsolve(Equations,X0); % or
[x,ssq,cnt] = LMFsolve(Equations,X0,'Name',Value,...); % or
[x,ssq,cnt] = LMFsolve(Equations,X0,Options) % .
In all cases, the applied variables have the following meaning:
* Equations is a function name (string) or a handle defining a set of equations,
* X0 is vector of initial estimates of solutions,
* x is the least-squares solution,
* ssq is sum of squares of equation residuals,
* cnt is a number of iterations

In the first case of call, default values of options are used. The second form of call defines selected options as a set of Name/Value pairs. The last alternative simplifies the statement by introducing earlier defined structure Options of Name\Value pairs.

Field names of the structure options are:
'Display' for control of iteration results,
'MaxIter' for setting maximum number of iterations,
'ScaleD' for defining diagonal matrix of scales,
'FunTol' for tolerance of final function values,
'XTol' for tolerance of final solution increments.

Example:
The general Rosenbrock's function has the form
f(x) = 100(x(2)-x(1)^2)^2 + (1-x(1))^2
Optimum solution gives f(x)=0 for x(1)=x(2)=1. Function f(x) can be expressed in the form
f(x) = f1(x)^2 + f2(x)^2, where f1(x) = 10(x(2)-x(1)^2) and f2(x) = 1-x(1).
Values of the functions f1(x) and f2(x) can be used as residuals.
LMFsolve finds the solution of this problem in 19 iterations. The more complicated problem sounds:
Find the least squares solution of the Rosenbrock valey inside a circle of the unit diameter centered at the origin. It is necessary to build third function, which is zero inside the circle and increasing outside it. This property has, say, the next penalty function:
f3(x) = sqrt(x(1)^2 + x(2)^2) - r, where r is a radius of the circle.
Its implementation using anonymous functions has the form
R = @(x) sqrt(x'*x)-.5; % A distance from the radius r=0.5
ros= @(x) [10*(x(2)-x(1)^2); 1-x(1); (R(x)>0)*R(x)*1000];
[x,ssq,cnt]=LMFsolve(ros,[-1.2,1],'Display',1,'MaxIter',50)
Solution: x = [0.4556; 0.2059], |x| = 0.5000
sum of squares: ssq = 0.2966,
number of iterations: cnt = 51.

Notes:
* Users with old MATLAB versions, which have no anonymous functions implemented, have to call LMFsolve with named function for residuals.
For above example it is
[x,ssq,cnt]=LMFsolve('rosen',[-1.2,1]);
where the function rosen.m is for the given problem of the form
function r = rosen(x)
% Rosenbrock's valey with a constraint R = sqrt(x(2)^2+x(1)^2)-.5;
% Residuals:
r = [10*(x(2)-x(1)^2) % first part
1-x(1) % second part
(R>0)*R*1000 % penalty
];
* The new version of the function LMFsolve is without erroneous part of analytical form of Jacobian matrix.
* The internal function printit.m has been replaced by the function of the same name taken from the more advanced function LMFnlsq (FEX Id 17534) because of much better form of output.
* An error causing an inclination of the previous version to instability has been removed. this step improved stability essentially, however, the a number of iterations increased, if the old version converged at all. However, much better behaviour has the full version of the Fletcher's algorithm, which is implemented in the function LMFnlsq (Id 17534).
* The old (unstable) version of the function is also inclided under the name LMFsolveOLD for those users who liked it.

Reference:
Fletcher, R., (1971): A Modified Marquardt Subroutine for Nonlinear Least Squares. Rpt. AERE-R 6799, Harwell

Miroslav Balda

### Miroslav Balda (view profile)

@Yan Li & others as well
It is clear that you don't read notes presented at the part "Comments and Ratings" and a help to the function or script you tried to use. There are all your questions answered. If you have read them carefully, you get to know:
1. The function LMFsolve is NOT maintained. Use the function LMFnlsq!
2. If you have questions, write me an e-mail and don't use "Comments and Ratings".
Miroslav Balda

Yan Li

### Yan Li (view profile)

It cannot work. It displays 'undefine inp function or variate

Sungtaek Cho

### Sungtaek Cho (view profile)

It's very powerful!
Thank you very much!!

Miroslav Balda

### Miroslav Balda (view profile)

@Rossella & others:
I have found your message only by chance. The function LMFsolve is NOT maintained. I announced already at Dec. 6, 2007 that it was replaced by function LMFnlsq. Please, use it because it is more stable and describes the way for stabilizing iterations by normalization of the set of unknowns. Moreover, the section you used for Comments & Ratings is improper for sending messages, for which is a direct way to the author by e-mail.

Rossella Stomati

### Rossella Stomati (view profile)

Hi, Prof.
thank you for the code, I have a question. I found this Warning using the function "Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 7.108829e-25. ". I think that this error can give me convergence problems. How can I solve?

Shrke Fu

### Shrke Fu (view profile)

Hi,Prof:
As a new learner of LMFsolve,I have a easy question .Can you explain the expression of
"r = [10*(x(2)-x(1)^2) % first part
1-x(1) % second part
(R>0)*R*1000 % penalty
]; "
I don't know the expression of "(R>0)*R*1000",what does that mean?

Timo Bonke

### Timo Bonke (view profile)

Hi,
can anyone tell me how to use 'ScaleD' correctly? Is this an option to limit the x values ?

AVINASH KUMAR

### AVINASH KUMAR (view profile)

LMFsave.m Do any1 have implemented the code for LM Method for QMF design..urgent help required

ahmed talib

### ahmed talib (view profile)

Hi every one
please any one can tell me how I used this package . I download the file but I cannot find the procedure to run the main file and put my data set and equation

Sam Fincham

### Sam Fincham (view profile)

Fantastic submission, thank you. : )

Miroslav Balda

### Miroslav Balda (view profile)

@Tomas
Sorry for late reply. I recommend you to use the function LMFnlsq or LMFnlsq2, that are also in File Exchange. The advantage is that both functions are maintained, while LMFsolve not. Both functions are also more stable and use almost equal data files. As far as any constraines is concerned, they may be solved by introduction of additional "penalty" residual for k-th unknown, say

(x(k)>30)*(x(k)-30)*w,

where w is a suitable weight, the value of which is found by experiment. If you have more troubles, you may send me your data and i'll try to solve it.

Tomas

### Tomas (view profile)

Dear Dr. Balda,
I've been trying to implement your code to a calibration problem I have with many variables. 9 fixed and one more for each measurement. I believe it is for this reason that your function has not been giving me consistent results and I end up iterating more than 1000 times. I already defined the funtions I want to minimize(the residuals) but I would like to add a restriction to some of the variables. The restriction would be something like I don't want to allow the absolut value of some of the x's to be greater than 30. How would you go about that?

Julien Piot

### Julien Piot (view profile)

Thank you, Professor Balda. Very useful.

Sandra Glisic

### Sandra Glisic (view profile)

Dear Dr Balda, I need help. Could you tell me can I use this LM application (algorithm) to sole a 6x6 matrix? (6 nonlinear equation, 6 variable to find 6 constant). Thank you in advance

Kristofer Kusano

jeyasenthil

Kevin

### Kevin (view profile)

Professor Balda:
I truly appreciate you for helping me out to solve non-linear equations. Your code beautifully worked out for me. Also, thanks again to correct matrix in your weekend time.
Dr. Balda, Thanks again for your great help and prompt support.
D. Kim

Hossein SOLEIMANI

### Hossein SOLEIMANI (view profile)

Hossein SOLEIMANI

### Hossein SOLEIMANI (view profile)

Hossein SOLEIMANI

### Hossein SOLEIMANI (view profile)

Hi

I want to solve a set of nonlinear equations that has a degree of freedom of 4 , and I want to use the experimental data in order to find the best results.
1. I have read the explanation about the FUN and I dont know how to put the funtions and also where I should include my experimental data?

regards

Miroslav Balda

### Miroslav Balda (view profile)

To Judith. What is wrong with your computer is difficult to find from your message. If you used the script LMFsolvetest, which is a part of the zipped file, you would get the solution. The stability has been reached in this case by a careful selection of optional parameters. It is written in the description, that the code is not perfectly stable and that it is better to use the function LMFnlsq (See the comment dated 06 Dec 2007).
If you try the old version (LMFsolveOLD), you would get also the answer. The solution of the problems could be solved much easier, if you read the last sentence in the section "Other requirements".

Judith Brands

### Judith Brands (view profile)

Analysing the example in the introduction of the file (Rosenbrock) gives different results on my computer as given in the example (cnt=7; ssq much larger). Furthermore, the results using this M-File are identical as to the results obtained with fminsearch? What goes wrong on my computer?

Nanang Firman

Ki-Young Shin

Wow!! I have been looking for LM algorithm (m file) for a long time.

Miroslav Balda

The function LMFsolve can be replaced by far more stable function LMFnlsq, FEX Id 17534

Miroslav Balda

The new extended description and help solves both notes.

Ilpo Nicholson

I cannot either understand the example (Rosenbrock function minimization with constraint) and the code (first & second part, penalty function). Maybe a some other curve fitting example would be more rewarding...?

Xie Gang

Excuse me, sir. I can't catch the command:
R = @(x) sqrt(x'*x)-.5; % A distance from the radius r=0.5
ros= @(x) [10*(x(2)-x(1)^2); 1-x(1); (R(x)>0)*R(x)*1000];
I use MATLAB6.5.
Thanks.

 11 Feb 2009 1.3.0.0 Improved description of the function behavior. Removed bugs in the function LMFsolveOLD names within the code. 9 Jan 2009 1.2.0.0 Removed part of analytical Jacobian matrix causing errors. Removed bug destabilizing the iteration process. Implemented new function for printing intermediate results. 9 Jan 2009 1.1.0.0 Removed erroneous part for analytical form of evaluation of Jacobian matrix. Introduced new function printit.m for better display of iteration results. Removed bug, which cased lower stability of the iteration process. 15 Nov 2007 1.0.0.0 Repaired description, complemented screnshot. A better solution, which is more stable, has been sent to FEX, see ID 17534. 8 Oct 2007 Extended description 4 Oct 2007 Improvements in description and help part 4 Oct 2007 improved description and help part of the code
##### MATLAB Release Compatibility
Created with R2006b
Compatible with any release
##### Platform Compatibility
Windows macOS Linux