The function is an improved version of the function LMFnlsq widely tested on the nonlinear regression, curve fitting and identification problems. Better description is in the complementing file LMFnlsq2test.pdf.

Warning: The option 'Display' is changed for better control of intermediate results output. See help LMFnlsq2.

I have tried my problem by myself. I find LMFnlsq2 is much better than other methods that I have tried.But among all 20 parameters to be optimized,two of them are always keep unaccurate, while others are good. What is the problem ?

Dear Balda, thank you for your excellent work.I want to use LMFnlsq2 to solve a complex cost function E,which involves as many as 20~50 variables. E is obtained from these variables through a series of transformations,such as affine,similarity and projective transformations.It cannot be described with one function,but 3~5 sub-functions.The jacobian matrix is cumbersome,but I can calculate its numerical solution.In this case,can the LMFnlsq2 work ? how should I use it ?

I'm trying to use the following method to find the intersecion area from 4 hyperbolas in which a mobile device is contained . Each hyperbola corresponds to a x, y coordinnate. However, I'm having some difficulty to prepare the inputs needed to use with this function. Can you help me please? Thank you.

Hi! thanks for the answer! I'm not quite sure I understand your comment, or maybe I didn't explained myself correctly. I have two images of size nxn who differ from each other. One is the result of applying a transformation, over the other.(let's say I2 is a transformed version of I1. every coordinate [x',y'] of I2 is the resulta of [t1, t2; t3 t4]*[x y]. Each coefficient ti is different for each pixel [x y] of I1).

Given I have prior knowledge over the domain I'm working on, I know that this transformation is very (e+-07~10) close to be the identity (i.e. the transformation is very "subtle", sort of speaking).
As I know this, I use as a first guess a matrix with nxn matrices (identities) of 2x2, which are the corresponding transformations for each pixel.

On each iteration, in my FUN, I apply the transformation over I1, and compare the result (square difference) with I2, hopefully reaching to I2.
I'm comparing images, not the transformation matrices. That's why I don't understand why it converges so rapidly, given that I1 and I2 differ by more than e-7, at least in the first iteration. Is it wrong that I use unity matrices as guesses?

I hope I made myself more clear this time
Thanks for the answers!

@Hugo
The reason why the iteration process finishes at the first step is probably in the fact that you have chosen the elements of matrix T as unknowns. Since they differ from unity by delta of order e-7, which is also a default value of the option FunTol, the iteration process finishes immediately, because it accepts the initial guess p0 as good enough for a solution.

I recommend to choose only differences from unity as unknowns, e.g. T(1,1)=1+p(1) and similarly T(2,2) = 1+p(2), where p's are the unknowns.
I guess that it will converge. In any case, will you inform me on the result?
Mira

Hello! I've a question regarding a problem I'm working on. I have two images X1 and X2 of size nxn, and a transformation matrix T with size nxn as well, where every component of T is a 2x2 matrix that "transform" every pixel of X1.
In my problem, I know that every matrix of T is very close to be the identity (difference of around ~e+-7). I want to use this method to, given X1 and X2, find out which is the transformation T that converted X1 in X2. I'm using T as a nxn matrix of 2x2 identities.
I'm not managing to get more than one iteration, because the method results in the identity as the better result even though I know that it isn't. I'll apreciate some advise in this regard. Thank you!

@Li
You may solve your problem as 3 independent examples, because there is no coupling among the equations. In that case, you build a body of your function FUN for residuals in the form
global x
res = fi(x,c) - yi;
where yi is the value of required fi.

Of course, you may to solve your problem also in one go, if you prepare the body of function FUN in the following way:
global x
res = [f1(x,c(1:2)) - y1
f2(x,c(3:4)) - y2
f3(x,c(5:6)) - y3];
This approach seems to be better, because you do not need to prepare 3 functions FUN.

If you have any issue with you problem, don't hesitate to contact me again or send me just data of your problem.

Dear Pro. Balda,
Does this file work with curve-fitting problems with several objective functions?
For example,
We obtain a set of experimental data: x(may be a vector),f1, f2, f3(f1, f2, f3 are scalars). We want to fit the following three euqations:
f1(x,c1,c2);f2(x,c3,c4);f3(x,c5,c6);
How should I implement the algorithm with LMFnlsq2?
Many thanks!

@Liu
The answer is trivial:
Let you have one set of 12 values of lefthand side, say in the column vector hmnp. Make a function for evaluation of differences (residuals) between righthand sides and lefthand sides, say resid.m in the form

function r = resid(x)
r = [x(1)+x(2)*p+ ... +(x(11)+x(12)*p)*n] - hmnp;

Here x(1) up to x(6) correspond C1 to C6 and x(7) up to x(12) correspon D1 up to D6. The simplest call of the function to solve the problem could be

[x,ssq,cnt] = LMFnlsq2('resid',x0);

A better way would be

[x,ssq,cnt] = LMFnlsq2('resid',x0,'Display',-50);

Of course that you have to supply a good estimate of the solution - coefficients C1, ..., D6. It is very important that you are able to do it, say from a nature of your experiment. The elements of x0 must not be zeros!

Since THe solution of your task has as many unknowns as equations, there is no
any degree of freedom. In consequence of it the solution should yield ssq->0.

Hi, Balda. Good day.
My curve fitting prblem is to calibrate the model like this:
h(m,n,p)=[C1+C2*P+(C3+C4*P)*m+(C5+C6*P)*n]/)[D1+D2*P+(D3+D4*P)*m+(D5+D6*P)*n]
The parameters to be estimate are C1,C2,C3,C4,C5,C6,D1,D2,D3,D4,D5,D6. And we have obtained 12 sets of data of h(m,n,p). So how to write the L-M expression using the LMFnlsq2.m ?

@Mihail
Yes, LMFnlsq2 is able to solve both real and complex problems, however, the complex task should be split into real and imagimary parts in the function for evaluating residuals. Thus, the dimension of the residual vector is double the number of equations, provided there are no additional conditions. The column vector of residuals should be real.

@xinxing
Build M-function, say resid.m, for evaluation of problem residuals:

function res = resid(p)
%xxxxxxxxxxxxxxxxxxxxxx
global t Y Wp % Wp = penalty weight
% If you wish to work with the native names of variables (not with the elements of the sought parameters p(i), use the following command:
[A,B,C1,C2,m,w,Tc] = deal2('_elm_',p); % deal2 is the function from the File Exchange
% http://www.mathworks.com/matlabcentral/fileexchange/32547
Tct = Tc - t;
lTt = log(Tct);
res = [A + B*Tct.^m + C1*cos(w*lTt) + C2*sin(w*lTt) - Y
(B>0)*B*Wp
(abs(C1)>1)*(abs(C1)-1)*Wp
(abs(C2)>1)*(abs(C2)-1)*Wp
(Tc<=max(t))*(Tc-max(t))*Wp
(m<0.1)*(m+0.1)*Wp
(m>0.9)*(m-0.9)*Wp
(w<6)(w+6)*Wp
(w>13)*(w-13)*Wp];

The user should choose propper weight(s) of penalties expressing constraints put on parameters.

Hi, Balda
Thanks for your reply. My curve fitting prblem is to calibrate the model like this:
Y(i)=A+B*(Tc-t(i))^m+C1*cos(w*ln(Tc-t(i))) +C2*sin(w*ln(Tc-t(i)))
The parameters to be estimate is A,B,C1,C2,m,w,Tc,
and the Constraints is:
B<0, |C1|<1, |C2|<1, Tc>max(t(i)), 0.1<m<0.9, 6<w<13， So how to write the L-M expression using the LMFnlsq2.m ?

Yes, you may make curve-fitting with constraints, if you introduce convenient penalty functions, which generate additional elements in the vector of residuals. It has been used in the solution of the Problem 3, Constrained Rosenbrock valey, described in the file LMFnlsq2test.m.

[A,v] = getAv(FUN,JAC,x,r,bdx);
%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SS = r'*r;

Updates

15 Apr 2013

1.1

Display of intermediate results has been improved to fulfill better users' requirements.

Thanks Jose's detailed study of the code, small bugs were repaired and the code made more clear.

16 Apr 2013

1.2

Repaired several bugs found by Jose. Improved description of the function and of the demo program LMFnlsq2test.

22 Apr 2013

1.3

The new version of the function is a slight modification of the recent update. One statement, dated 2012-12-01, has been commented out because of its erroneous function under certain circumstances. The function is now fast and much more stable.

23 Apr 2013

1.6

Improved the printout of results. The previous versions did not display the final solution if the iteration count were not integer multiple of required step of printing.