Path: news.mathworks.com!not-for-mail
From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Finding the minimimum of a 2D surface
Date: Mon, 10 May 2010 03:28:04 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 38
Message-ID: <hs7ug4$a96$1@fred.mathworks.com>
References: <hs7kqn$s12$1@fred.mathworks.com>
Reply-To: <HIDDEN>
NNTP-Posting-Host: webapp-05-blr.mathworks.com
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1273462084 10534 172.30.248.35 (10 May 2010 03:28:04 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Mon, 10 May 2010 03:28:04 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1187260
Xref: news.mathworks.com comp.soft-sys.matlab:634101

"CFY30 CFY30" <cfy30@yahoo.com> wrote in message <hs7kqn$s12$1@fred.mathworks.com>...
> Hi all,
> 
> I am trying to find the minimum of a 2D surface, (x-4)^2/a + y^2/b by gradient search. The code I have is as follows. I expect x should converge to 4 and y should converge to 0 but both of them converge to 2. Can anyone help what is wrong with my code?
> 
> Thanks,
> cfy30
> 
> a = 4;
> b = 4;
> mu    = 0.01;
> x_old = 0;
> x_new = 1;
> y_old = 0;
> y_new = 1;
> z_old = 0;
> for n=1:100000
>     z_new      = (x_new-4)^2/a + y_new^2/b;
>     dz         = z_new - z_old;
>     dx         = x_new - x_old;
>     dy         = y_new - y_old;
>     if ((abs(dz)<1e-12))
>         break;
>     end;
>     x_old      = x_new;
>     y_old      = y_new;
>     z_old      = z_new;
>     x_new      = x_new - mu*dz/dx;
>     y_new      = y_new - mu*dz/dy;
> end;
- - - - - - -
  This algorithm is fundamentally flawed in my opinion.  The quantities you call dz/dx and dz/dy are apparently intended to be approximate gradient components.  However, gradient values are obtained by varying x and y independently from one another to determine their separate effects on z.  That is the definition of partial derivatives - they are rates of changes with respect to independent variable changes.  Here you are only varying x and y in a single way and attempting to judge the gradient components as being inversely proportional to the sizes of the x and y changes, and not from independent effects.  Therefore you are not really following in the direction of the downhill gradient.

  Even worse, your first amounts of change for x and y were equal for each one.  The effect is therefore that subsequent changes in x and y must remain forever equal and you are locked into moving along a straight line x-y = 0, and that is consistent with your erroneous (2,2) result.

  However, I suspect you would have serious troubles even with a different starting direction.  Instead you need to alter the basic nature of the algorithm to calculate a much more realistic gradient direction.

Roger Stafford