From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Non-linear optimization
Date: Thu, 7 Mar 2013 21:36:08 +0000 (UTC)
Organization: Xoran Technologies
Lines: 18
Message-ID: <khb188$n3q$>
References: <kh2m44$4eh$> <kh32ds$hrn$> <kh46mg$s0t$> <kh89he$4al$> <kh8bsc$bov$> <kh8dvs$imu$> <kh8g3b$p6i$> <kh8hi7$t86$> <kh8jcc$4qj$> <kh8skk$s6c$> <kh9it4$qg1$> <khaui3$efe$> <khavr6$imm$>
Reply-To: <HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: 1362692168 23674 (7 Mar 2013 21:36:08 GMT)
NNTP-Posting-Date: Thu, 7 Mar 2013 21:36:08 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1440443
Xref: comp.soft-sys.matlab:790595

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <khavr6$imm$>...
> "Matt J" wrote in message <khaui3$efe$>...
> > 
> > I'm not sure what link with quasi-Newton that you're referring to. If you're saying that
> > 
> >  (H+lambda*I) x=gradient
> > 
> > is an LM generalization of Newton's method, 
> Quasi-Newton -> Replace Hessian by an appoximation of it, usually based from the first derivative, such as BFGS formula or H ~ J'*J in the least square cost function, where J is the Jacobian of the model.
> >then yes, I'm sure Newton-LM would converge faster, however each iteration looks costly. You would have to know the minimum eigenvalue of H in order to make (H+lambda*I) positive definite.
> Both BFGS and J'*J approximation provide quasi convex quadratic approximation. Therefore there is no need to bother with such detail about positiveness.

That much, I understand. Maybe I didn't understand what you meant by quasi-Newton being "quasi-efficient". It looks like finding lambda for quasi-Newton-LM would be much more efficient than for true Newton-LM.