Path: news.mathworks.com!not-for-mail
From: "John D'Errico" <woodchips@rochester.rr.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: [SOLVED] Re: How to use optimset for fminsearch to run Levenberg-Marquardt instead of Nelder-Mead?
Date: Thu, 29 Oct 2009 15:20:20 +0000 (UTC)
Organization: John D'Errico (1-3LEW5R)
Lines: 31
Message-ID: <hccbrk$87m$1@fred.mathworks.com>
References: <hcbn39$k4a$1@fred.mathworks.com> <hcc19l$neo$1@fred.mathworks.com> <hccahp$d2b$1@fred.mathworks.com>
Reply-To: "John D'Errico" <woodchips@rochester.rr.com>
NNTP-Posting-Host: webapp-05-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1256829620 8438 172.30.248.35 (29 Oct 2009 15:20:20 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Thu, 29 Oct 2009 15:20:20 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 869215
Xref: news.mathworks.com comp.soft-sys.matlab:581008


"Magnus " <magax@itn.liu.se> wrote in message <hccahp$d2b$1@fred.mathworks.com>...
> Update:
> 
> In the little test I mentioned previously I tried to determine if fminsearch could use Levenberg-Marquardt instead of Nelder-Mead. I did this by including and excluding the following options as argument to the fminsearch.
> 
> options = optimset('MaxFunEvals', 10000, 'MaxIter', 10000);
> options = optimset(options, 'LevenbergMarquardt, 'on')
> 
> I found that without these options fminsearch quite prematurely with flag 0 after approx 1400 iterations but with the options included fminsearch completed after some 2000 iterations. Because the results were different I prematurely concluded that Levenberg-Marquardt was used instead of Nelder-Mead. This is not the case as I was wrong on two accounts:
> 
> Firstly, by looking in the fminsearch.m code I can see that: a)  Line 78: The default number of MaxIter and MaxFunEvals is 200*numberOfVariables, b) Line 111-112: numberOfVariables is defined as the number of elements in the parameter array.
> 
> The reason why fminsearch quit prematurely at 1400 iterations was not because of a different optimization algorithm but because the MaxFunEvals had been exceeded. MaxFunEvals is not called once for every iteration, thus it was larger than numberOfVariables * 200, which in my case with 10 parameters is 2000.
> 
> Secondly, in changing the options to use the Levenberg-Marquardt algorithm I also allowed for more iterations and more function evaluations. It is the change in these variables that cause the fminsearch to eventually reach a minima. 
> 
> This is further supported by c) Line 417: Only here is ouput.algorithm set to any value that would specify if any other algorithm is used.
> 
> Conclusion: fminsearch only uses Nelder-Mead and ignores 'LevenbergMarquardt', 'on' in the optimset input argument. No warning or error is given.

There is no reason to assume that fminsearch will use
a wildly different algorithm that has not been coded
into it. Fminsearch is a Nelder/Mead optimizer, to
minimize a general function. LM is a method that is
specifically for nonlinear least squares problems.

In fact, there is no reason for fminsearch to even test
to see if you have set some parameters that it will never
look at. In fact, it does not do so.

John