Path: news.mathworks.com!not-for-mail From: <HIDDEN> Newsgroups: comp.soft-sys.matlab Subject: Re: Non-linear optimization Date: Thu, 7 Mar 2013 21:36:08 +0000 (UTC) Organization: Xoran Technologies Lines: 18 Message-ID: <khb188$n3q$1@newscl01ah.mathworks.com> References: <kh2m44$4eh$1@newscl01ah.mathworks.com> <kh32ds$hrn$1@newscl01ah.mathworks.com> <kh46mg$s0t$1@newscl01ah.mathworks.com> <kh89he$4al$1@newscl01ah.mathworks.com> <kh8bsc$bov$1@newscl01ah.mathworks.com> <kh8dvs$imu$1@newscl01ah.mathworks.com> <kh8g3b$p6i$1@newscl01ah.mathworks.com> <kh8hi7$t86$1@newscl01ah.mathworks.com> <kh8jcc$4qj$1@newscl01ah.mathworks.com> <kh8skk$s6c$1@newscl01ah.mathworks.com> <kh9it4$qg1$1@newscl01ah.mathworks.com> <khaui3$efe$1@newscl01ah.mathworks.com> <khavr6$imm$1@newscl01ah.mathworks.com> Reply-To: <HIDDEN> NNTP-Posting-Host: www-04-blr.mathworks.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Trace: newscl01ah.mathworks.com 1362692168 23674 172.30.248.35 (7 Mar 2013 21:36:08 GMT) X-Complaints-To: news@mathworks.com NNTP-Posting-Date: Thu, 7 Mar 2013 21:36:08 +0000 (UTC) X-Newsreader: MATLAB Central Newsreader 1440443 Xref: news.mathworks.com comp.soft-sys.matlab:790595 "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <khavr6$imm$1@newscl01ah.mathworks.com>... > "Matt J" wrote in message <khaui3$efe$1@newscl01ah.mathworks.com>... > > > > > I'm not sure what link with quasi-Newton that you're referring to. If you're saying that > > > > (H+lambda*I) x=gradient > > > > is an LM generalization of Newton's method, > > Quasi-Newton -> Replace Hessian by an appoximation of it, usually based from the first derivative, such as BFGS formula or H ~ J'*J in the least square cost function, where J is the Jacobian of the model. > > >then yes, I'm sure Newton-LM would converge faster, however each iteration looks costly. You would have to know the minimum eigenvalue of H in order to make (H+lambda*I) positive definite. > > Both BFGS and J'*J approximation provide quasi convex quadratic approximation. Therefore there is no need to bother with such detail about positiveness. ======== That much, I understand. Maybe I didn't understand what you meant by quasi-Newton being "quasi-efficient". It looks like finding lambda for quasi-Newton-LM would be much more efficient than for true Newton-LM.