From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Best fit line with constrained coefficients
Date: Tue, 11 May 2010 19:23:05 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 27
Message-ID: <hscaqo$g36$>
References: <hs9pck$618$> <hsbs46$q21$>
Reply-To: <HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: 1273605785 16486 (11 May 2010 19:23:05 GMT)
NNTP-Posting-Date: Tue, 11 May 2010 19:23:05 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1187260
Xref: comp.soft-sys.matlab:634733

"Nathan " <> wrote in message <hsbs46$q21$>...
> Is there a way to use orthogonal offsets instead of vertical offsets when doing the least squares fitting?  I think that may be more of what I am envisioning.
> .........

To Nathan:

  There is nothing difficult about finding orthogonal fits.  For your data x and y you would do this:

 xm = mean(x); ym = mean(y);
 A = [x-xm;y=ym].';
 [U,S,V] = svd(A,0);

Then V(:,1) is a unit vector parallel to the desired line, so its slope is

 m = V(2,1)/V(1,1)

and the line runs through the point (xm,ym).

  The assumption behind ordinary regression is that all the errors are in the y-ccordinates, while these orthogonal fits assume that both x and y coordinates are equally in error.  You can revise the above to weight the respective coordinates so as to obtain an optimum line in the sense of correspondingly weighted least square errors.

  I seem to recall that the orthogonal-fit slope wil always lie between that of the ordinary regression line on the one hand and the regression line with x and y interchanged on the other, and that the three slopes can only be equal when the original data is colinear.  (I would have to brush off some mental cobwebs to prove this right now.)

To Matt J:

  When you write y(x) = m^2*x + b and remove all constraints, the optimization procedure will move toward a solution in which the partial derivatives of the objective function with respect to m and b are zero, since there is no longer a constraint barrier to stop it.  If the data is such that it cannot find such a solution with m^2 > 0 - in other words, the natural regression slope would be negative - then it will presumably gravitate (if its search is successful) toward the m = 0 value which will then be the only way it can achieve a zero partial derivative with respect to m.  That was the basis of my statement earlier.

Roger Stafford