Path: news.mathworks.com!not-for-mail
From: "Derya Ozyurt" <dozyurt@mathworks.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: [SOLVED] Re: How to use optimset for fminsearch to run Levenberg-Marquardt instead of Nelder-Mead?
Date: Tue, 3 Nov 2009 13:33:20 -0500
Organization: The MathWorks, Inc.
Lines: 84
Message-ID: <hcpt1i$h4h$1@fred.mathworks.com>
References: <hcbn39$k4a$1@fred.mathworks.com> <hcc19l$neo$1@fred.mathworks.com>
Reply-To: "Derya Ozyurt" <dozyurt@mathworks.com>
NNTP-Posting-Host: ozyurtd.dhcp.mathworks.com
X-Trace: fred.mathworks.com 1257273202 17553 172.31.56.82 (3 Nov 2009 18:33:22 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Tue, 3 Nov 2009 18:33:22 +0000 (UTC)
X-Priority: 3
X-MSMail-Priority: Normal
X-Newsreader: Microsoft Outlook Express 6.00.2900.5843
X-RFC2646: Format=Flowed; Original
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579
Xref: news.mathworks.com comp.soft-sys.matlab:582124


Hi Magnus,
fminsearch is the implementation of Nelder-Mead simplex method only. Could 
you confirm that by only changing an option (LevenbergMarquardt) that is not 
applicable to fminsearch, you obtained a different result for your problem? 
Your problem doesn't have any randomness, right?

Best Regards,
Derya

"Magnus " <magax@itn.liu.se> wrote in message 
news:hcc19l$neo$1@fred.mathworks.com...
> Update:
>
> Following a recommendation of a colleague I tried the fminsearch on an 
> identical set of data for the two cases when optimset was set to use 
> Nelder-Mead (NM) and when it was set to Levenberg-Marquardt (LM). The 
> output shows that in the NM case the optimizer never finishes but quits 
> with flag 0 after 1406 iterations. In the LM case the optimizer quite 
> successfully with flag 1 with 2241 iterations.
>
> This would imply that the optimset argument does make a difference but 
> that the algorithm field in the output structure still reads "Nelder-mead 
> simplex direct search" even if it in fact is using LM.
>
> /M.
>
>
>
> "Magnus " <magax@itn.liu.se> wrote in message 
> <hcbn39$k4a$1@fred.mathworks.com>...
>> Dear All,
>>
>> I have searched the newsgroup for "fminsearch optimset" with combinations 
>> on "Levenberg-Marquardt" but I am still not finding what I am looking 
>> for. I hope you can help.
>>
>> Question: I am suspecting that fminsearch cannot be changed to use any 
>> other optimization algorithm than Nelder-Mead and that my changes to the 
>> optimset argument are simply ignored. Is this true? If not, any 
>> suggestions on how I can change fminsearch to work with 
>> Levenberg-Marquardt? Alternatively, what other Matlab function would be 
>> appropriate for this task?
>>
>> Application: I am trying to estimate a projection matrix and I want to 
>> use fminsearch to find the optimal parameters to minimize the 
>> reprojection error, i.e. the screen distance between some ground truth 
>> markers projected on the screen and some additional test markers also 
>> projected to the screen.
>>
>> My code, using Matlab 7.3.0.298 (R2006b) on Ubuntu Dapper Drake, looks 
>> like this:
>>
>> % Here I try to change the fminsearch's default optimiser from 
>> Nelder-Mead to Levenberg-Marquardt and also make it run for a little 
>> longer
>> options = optimset('MaxFunEvals', 10000, 'MaxIter', 10000);
>> options = optimset(options, 'LevenbergMarquardt, 'on');
>>
>> % Here I call fminsearch with a function handle to the function that 
>> calculates an error, an array of parameter estimates that are to be 
>> changed such that the error returned by the function in the function 
>> handle is minimized. This error is reported back to errval, and I can 
>> also see if the optimizer terminated prematurely in flag. The output data 
>> structure also gives me further information.
>> [estimates errval flag output] = fminsearch(@psuedoHuberCostFunction, [rx 
>> ry rz tx ty tz fx fy px py], options);
>>
>> Observations:
>>
>> 1. The code as written here above runs, but the output.algorithm, which 
>> indicates which algorithm that was used, says that Nelder Mead was used.
>> 2. If I change the call to optimset to any other string than 
>> 'LevenbergMarquardt', this results in an error.
>>
>> Motivation: I want to see if Levenberg-Marquardt with its adaptable step 
>> length can deal with a more complex parameter surface compared to 
>> Nelder-Mead and therefore is a better choice for camera calibrations with 
>> some alignment noise between correspondence points.
>>
>> Thanks for your time and help.
>>
>> Magnus