Thread Subject: Weight-bug in curve fitting toolbox?

Dear readers,
I have recently been looking through the options in Matlab R2008a for performing a weighted robust linear regression. It seems that the curve fitting toolbox is the only function that accepts additional weights. However, I might have come across some inconsistency in the usage of the weights that I would like to have confirmed.

In the help section on Robust Least Squares (Least-Squares Fitting, Curve Fitting Toolbox) it says the following: "Note that if you supply your own regression weight vector, the final weight is the product of the robust weight and the regression weight".

In the code of fit.m there is no such multiplication, in fact the weights from the robust fit are not eve given as outputs. What I did find though is that the initially supplied weights are squared on the first line in the "goodstruct"-function. Is this really correct or is this supposed to be the product of the robust and initial weights as stated in the help section? Unfortunately this calculation will affect all the subsequent calculations of the statistics so at the moment I don't really trust in using the cft for this purpose.

I have checked the code in Matlab 2009a as well and it is the same thing. I would really appreciate it if someone could tell me if this is correct or if this is a bug that has slipped through every revision. Thanks!

Hi Basti,

What makes you suspect that there is a bug?

> In the help section on Robust Least Squares (Least-Squares
> Fitting, Curve Fitting Toolbox) it says the following:
> "Note that if you supply your own regression weight vector,
> the final weight is the product of the robust weight and the
> regression weight".
>
> In the code of fit.m there is no such multiplication…

The multiplication is there. In fact, it is there twice: once for linear fitting and once for non-linear.

For non-linear fitting, look at matlab\toolbox\curvefit\curvefit\private\cfrobnlinfit.m

Line 97: wts.*bw
   wts = user supplied weights
   bw = robust weights

For linear fitting, inside FIT.m, cfroblinfit gets called with y already weighted and then calls linearfit with the robust weights, bw. Therefore the multiplication is on line 844 of fit.m:

    Dy = sqrtw.*y;

Line numbers are to the R2009a versions. They may different in other versions.

Cheers,
--Jon

"Jon Cherrie" <jcherrie.n.o.s.p.a.m.@mathworks.co.uk> wrote in message <h31pbu$6s8$1@fred.mathworks.com>...
> Hi Basti,
>
> What makes you suspect that there is a bug?
>
> > In the help section on Robust Least Squares (Least-Squares
> > Fitting, Curve Fitting Toolbox) it says the following:
> > "Note that if you supply your own regression weight vector,
> > the final weight is the product of the robust weight and the
> > regression weight".
> >
> > In the code of fit.m there is no such multiplication…
>
> The multiplication is there. In fact, it is there twice: once for linear fitting and once for non-linear.
>
> For non-linear fitting, look at matlab\toolbox\curvefit\curvefit\private\cfrobnlinfit.m
>
> Line 97: wts.*bw
> wts = user supplied weights
> bw = robust weights
>
> For linear fitting, inside FIT.m, cfroblinfit gets called with y already weighted and then calls linearfit with the robust weights, bw. Therefore the multiplication is on line 844 of fit.m:
>
> Dy = sqrtw.*y;
>
> Line numbers are to the R2009a versions. They may different in other versions.
>
> Cheers,
> --Jon

Hey Jon,
Thanks for your answer. You are right that the multiplication of the weights is done in the regression, but cfroblinfit doesn't give the final weights as an output. My concern is therfore with the goodstruct-function where all the statistics are calculated. This function is called on line 571 (fit.m, linear model, R2009a) with:

goodness = goodstruct(ydata,weights,res,dfe,output,sse);

As far as I can see the variable weights has not been changed and is therefore the initial weights. In R2008a there is even an additional squaring of the weights on the first line of the goodstruct-function which I don't understand the reason for:

              weightsfinal = weights.*weights;

Perhaps I am wrong and the statistics should be calculated with only the initial weights, but it doesn't seem right since possible outliers will be taken into account then and the statistics won't represent the actual regression. Please correct me if I'm wrong.

Best,
Basti