Thread Subject:

Estimation of variables to minimize the sum of residuals

Hi everyone

Here is the problem which I am working on:
***********************************************
Let's say we have an array x = [1; 2; 3; 4]. Now, lets define a function f = (a + b*y + c*y.^2) where y is an array with same size as x. (e.g: y = [-2; -1; 0; 2])

Define Residual:

res = x - f
(e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).

Define sum of square of residuals as:

s = sum(res.^2).

Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
*****************************************************
My approach:

I defined residual as an inline function

x = [1; 2; 3; 4]
y = [-2; -1; 0; 2]
res = inline('x - (a + b*y + c*y.^2), 'x', 'y', 'a', 'b', 'c');
var = @(var) f(x, y, var(1), var(2), var(3));
*****************************

What i am not able to code is how to estimate the sum of the residual and then how to find values of a, b, c such that this sum is minimum. I know that there is a function called fminsearch...but i am not able to apply this here. Any help with be appreciated

Thanks a lot!

Regards
Prakhar (PhD student, Caltech)

"Prakhar " <prakhar_cool@yahoo.com> wrote in message <k79nrs$oss$1@newscl01ah.mathworks.com>...
> res = x - f
> (e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).
> Define sum of square of residuals as:
> s = sum(res.^2).
> Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
- - - - - - - - -
  This is a simple problem in linear regression. We assume x and y are column vectors as shown. Do:

 abc = [ones(size(y)),y,y.^2]\x;

where 'abc' will be a three-element column vector containing the optimum a, b, and c coefficients in that order.

Roger Stafford

"Prakhar " <prakhar_cool@yahoo.com> wrote in message <k79nrs$oss$1@newscl01ah.mathworks.com>...
> Hi everyone
>
> Here is the problem which I am working on:
> ***********************************************
> Let's say we have an array x = [1; 2; 3; 4]. Now, lets define a function f = (a + b*y + c*y.^2) where y is an array with same size as x. (e.g: y = [-2; -1; 0; 2])
>
> Define Residual:
>
> res = x - f
> (e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).
>
> Define sum of square of residuals as:
>
> s = sum(res.^2).
>
> Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
> *****************************************************
> My approach:
>
> I defined residual as an inline function
>
> x = [1; 2; 3; 4]
> y = [-2; -1; 0; 2]
> res = inline('x - (a + b*y + c*y.^2), 'x', 'y', 'a', 'b', 'c');
> var = @(var) f(x, y, var(1), var(2), var(3));
> *****************************
>
> What i am not able to code is how to estimate the sum of the residual and then how to find values of a, b, c such that this sum is minimum. I know that there is a function called fminsearch...but i am not able to apply this here. Any help with be appreciated

P = polyfit(y, x, 2);
c =P(1); b = P(2); a = P(3);

% Bruno

"Prakhar " <prakhar_cool@yahoo.com> wrote in message <k79nrs$oss$1@newscl01ah.mathworks.com>...
> Hi everyone
>
> Here is the problem which I am working on:
> ***********************************************
> Let's say we have an array x = [1; 2; 3; 4]. Now, lets define a function f = (a + b*y + c*y.^2) where y is an array with same size as x. (e.g: y = [-2; -1; 0; 2])
>
> Define Residual:
>
> res = x - f
> (e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).
>
> Define sum of square of residuals as:
>
> s = sum(res.^2).
>
> Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
> *****************************************************
> My approach:
>
> I defined residual as an inline function
>
> x = [1; 2; 3; 4]
> y = [-2; -1; 0; 2]
> res = inline('x - (a + b*y + c*y.^2), 'x', 'y', 'a', 'b', 'c');
> var = @(var) f(x, y, var(1), var(2), var(3));
> *****************************
>
> What i am not able to code is how to estimate the sum of the residual and then how to find values of a, b, c such that this sum is minimum. I know that there is a function called fminsearch...but i am not able to apply this here. Any help with be appreciated
>
> Thanks a lot!
>
> Regards
> Prakhar (PhD student, Caltech)

Easiest way is to use lsqcurvefit function in optimization toolbox. I am pretty sure Caltech have license for optimization toolbox.

Here is what I did on your test example. Assume, k = [a b c]
y = [-2; -1; 0; 2]; % typically called 'xdata' in lsqcurvefit help pages
x = [1; 2; 3; 4]; % typically called 'ydata'
% define model
fun = @(k,y) k(1) + k(2)*y + k(3)*y.^2
% Fit model by minimizing the least square of the residual
[o1,o2,o3] = lsqcurvefit(fun,[1 2 3], y, x)

Look at the example at the bottom of this page:
http://www.mathworks.com/help/optim/ug/lsqcurvefit.html

Thank you Rakesh!. Got it working. :)

-Prakhar

"Rakesh Kumar" wrote in message <k7b8ag$4jn$1@newscl01ah.mathworks.com>...
> "Prakhar " <prakhar_cool@yahoo.com> wrote in message <k79nrs$oss$1@newscl01ah.mathworks.com>...
> > Hi everyone
> >
> > Here is the problem which I am working on:
> > ***********************************************
> > Let's say we have an array x = [1; 2; 3; 4]. Now, lets define a function f = (a + b*y + c*y.^2) where y is an array with same size as x. (e.g: y = [-2; -1; 0; 2])
> >
> > Define Residual:
> >
> > res = x - f
> > (e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).
> >
> > Define sum of square of residuals as:
> >
> > s = sum(res.^2).
> >
> > Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
> > *****************************************************
> > My approach:
> >
> > I defined residual as an inline function
> >
> > x = [1; 2; 3; 4]
> > y = [-2; -1; 0; 2]
> > res = inline('x - (a + b*y + c*y.^2), 'x', 'y', 'a', 'b', 'c');
> > var = @(var) f(x, y, var(1), var(2), var(3));
> > *****************************
> >
> > What i am not able to code is how to estimate the sum of the residual and then how to find values of a, b, c such that this sum is minimum. I know that there is a function called fminsearch...but i am not able to apply this here. Any help with be appreciated
> >
> > Thanks a lot!
> >
> > Regards
> > Prakhar (PhD student, Caltech)
>
> Easiest way is to use lsqcurvefit function in optimization toolbox. I am pretty sure Caltech have license for optimization toolbox.
>
> Here is what I did on your test example. Assume, k = [a b c]
> y = [-2; -1; 0; 2]; % typically called 'xdata' in lsqcurvefit help pages
> x = [1; 2; 3; 4]; % typically called 'ydata'
> % define model
> fun = @(k,y) k(1) + k(2)*y + k(3)*y.^2
> % Fit model by minimizing the least square of the residual
> [o1,o2,o3] = lsqcurvefit(fun,[1 2 3], y, x)
>
> Look at the example at the bottom of this page:
> http://www.mathworks.com/help/optim/ug/lsqcurvefit.html

Thanks Bruno for your reply. Appreciate it.

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <k7ada4$34h$1@newscl01ah.mathworks.com>...
> "Prakhar " <prakhar_cool@yahoo.com> wrote in message <k79nrs$oss$1@newscl01ah.mathworks.com>...
> > Hi everyone
> >
> > Here is the problem which I am working on:
> > ***********************************************
> > Let's say we have an array x = [1; 2; 3; 4]. Now, lets define a function f = (a + b*y + c*y.^2) where y is an array with same size as x. (e.g: y = [-2; -1; 0; 2])
> >
> > Define Residual:
> >
> > res = x - f
> > (e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).
> >
> > Define sum of square of residuals as:
> >
> > s = sum(res.^2).
> >
> > Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
> > *****************************************************
> > My approach:
> >
> > I defined residual as an inline function
> >
> > x = [1; 2; 3; 4]
> > y = [-2; -1; 0; 2]
> > res = inline('x - (a + b*y + c*y.^2), 'x', 'y', 'a', 'b', 'c');
> > var = @(var) f(x, y, var(1), var(2), var(3));
> > *****************************
> >
> > What i am not able to code is how to estimate the sum of the residual and then how to find values of a, b, c such that this sum is minimum. I know that there is a function called fminsearch...but i am not able to apply this here. Any help with be appreciated
>
> P = polyfit(y, x, 2);
> c =P(1); b = P(2); a = P(3);
>
> % Bruno

Thanks Roger. Yeah...i figured it out thats its simple linear regression. Thanks for prompting it out.

-Prakhar

"Roger Stafford" wrote in message <k79vn5$kdj$1@newscl01ah.mathworks.com>...
> "Prakhar " <prakhar_cool@yahoo.com> wrote in message <k79nrs$oss$1@newscl01ah.mathworks.com>...
> > res = x - f
> > (e.g.: res(1) = x(1) - (a + b*y(1) + c*y(1)^2).
> > Define sum of square of residuals as:
> > s = sum(res.^2).
> > Now, i need to find variables a, b, c in such a way so that the sum of residuals 's' is minimum.
> - - - - - - - - -
> This is a simple problem in linear regression. We assume x and y are column vectors as shown. Do:
>
> abc = [ones(size(y)),y,y.^2]\x;
>
> where 'abc' will be a three-element column vector containing the optimum a, b, and c coefficients in that order.
>
> Roger Stafford