Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Covariance matrix from a single regression

Subject: Covariance matrix from a single regression

From: Firat

Date: 14 Jun, 2010 04:00:25

Message: 1 of 6

Hi all,
I am trying to fit a linear model to a set of data. F=G*K is my equation, where I know G and F data and try to estimate K. I have 240x1 F matrix, a G matrix of size 240x4 and K matrix should be 4x1. I can fit a linear model to that data and get the K matrix. My question is: Is there a way to get the covariance matrix of that single model ?



I hope it is clear,
Thank you,
Firat

Subject: Covariance matrix from a single regression

From: Firat

Date: 14 Jun, 2010 04:47:03

Message: 2 of 6

Actually I should have said multiple regression;
F= K1*G1 + K2 *G2 +K3*G3 + K4*G4

I am trying to get the statistics of the K terms basically.

Thanks,
Firat

"Firat " <firateren85@gmail.com> wrote in message <hv49go$sm$1@fred.mathworks.com>...
> Hi all,
> I am trying to fit a linear model to a set of data. F=G*K is my equation, where I know G and F data and try to estimate K. I have 240x1 F matrix, a G matrix of size 240x4 and K matrix should be 4x1. I can fit a linear model to that data and get the K matrix. My question is: Is there a way to get the covariance matrix of that single model ?
>
>
>
> I hope it is clear,
> Thank you,
> Firat

Subject: Covariance matrix from a single regression

From: Roger Stafford

Date: 14 Jun, 2010 08:34:03

Message: 3 of 6

"Firat " <firateren85@gmail.com> wrote in message <hv49go$sm$1@fred.mathworks.com>...
> Hi all,
> I am trying to fit a linear model to a set of data. F=G*K is my equation, where I know G and F data and try to estimate K. I have 240x1 F matrix, a G matrix of size 240x4 and K matrix should be 4x1. I can fit a linear model to that data and get the K matrix. My question is: Is there a way to get the covariance matrix of that single model ?
> Actually I should have said multiple regression;
> F= K1*G1 + K2 *G2 +K3*G3 + K4*G4
> I am trying to get the statistics of the K terms basically.
> .......

  With multiple regression the precise result you would get for K is:

 K = G\F;

using the matlab backslash operator. However, you are asking about the statistical properties of such a result, with the apparent implication that you would like to obtain at least an estimate of these properties from the data present in F and G themselves. This is a far more difficult task.

  This is analogous to the simpler estimate of the variance of a single random variable based on n observations of it. In this case the estimate that is commonly used is to subtract the mean of the observed values from each value, add the squares of these differences, and then divide by n-1 rather than n to make it "unbiased".

  I confess I do not know what a reasonable estimate would be for the statistical properties of G\F, but it would certainly involve such things as the observed four dimensional distribution of the rows of G, as well as correlations of F values with these rows.

  I would also guess further that far more than 240 samples would be needed to arrive at a reliable estimate of these properties, simply because it requires a great many points to thoroughly explore all the variations inherent in a four-dimensional situation. For example if a hundred samples are considered as necessary to get a good estimate of a single variable's distribution, then about a hundred to the fourth power, or a hundred million, would be necessary to thoroughly explore the corresponding four dimensions.

  All this is by way of saying that in my opinion you have tackled a real "bear" with the question you have asked here. Of course an easy way out is to repeat the set of 240 observations many times independently and calculate the variances and covariances of the different quadruples of G\F that are produced. However, that seems a little frustrating, since it says, "if I had made only 240 observations, these would be the uncertainties in my answer." But with a lot more than 240 actually available, why not make a more reliable estimate with G and F that much larger?

Roger Stafford

Subject: Covariance matrix from a single regression

From: TideMan

Date: 14 Jun, 2010 08:48:24

Message: 4 of 6

On Jun 14, 8:34 pm, "Roger Stafford"
<ellieandrogerxy...@mindspring.com.invalid> wrote:
> "Firat " <firatere...@gmail.com> wrote in message <hv49go$s...@fred.mathworks.com>...
> > Hi all,
> > I am trying to fit a linear model to a set of data.  F=G*K is my equation, where I know G and F data and try to estimate K. I have 240x1 F matrix, a G matrix of size 240x4 and K matrix should be 4x1. I can fit a linear model to that data and get the K matrix. My question is: Is there a way to get the covariance matrix of that single model ?
> > Actually I should have said multiple regression;
> > F= K1*G1 + K2 *G2 +K3*G3 + K4*G4
> > I am trying to get the statistics of the K terms basically.
> > .......
>
>   With multiple regression the precise result you would get for K is:
>
>  K = G\F;
>
> using the matlab backslash operator.  However, you are asking about the statistical properties of such a result, with the apparent implication that you would like to obtain at least an estimate of these properties from the data present in F and G themselves.  This is a far more difficult task.
>
>   This is analogous to the simpler estimate of the variance of a single random variable based on n observations of it.  In this case the estimate that is commonly used is to subtract the mean of the observed values from each value, add the squares of these differences, and then divide by n-1 rather than n to make it "unbiased".
>
>   I confess I do not know what a reasonable estimate would be for the statistical properties of G\F, but it would certainly involve such things as the observed four dimensional distribution of the rows of G, as well as correlations of F values with these rows.
>
>   I would also guess further that far more than 240 samples would be needed to arrive at a reliable estimate of these properties, simply because it requires a great many points to thoroughly explore all the variations inherent in a four-dimensional situation.  For example if a hundred samples are considered as necessary to get a good estimate of a single variable's distribution, then about a hundred to the fourth power, or a hundred million, would be necessary to thoroughly explore the corresponding four dimensions.
>
>   All this is by way of saying that in my opinion you have tackled a real "bear" with the question you have asked here.  Of course an easy way out is to repeat the set of 240 observations many times independently and calculate the variances and covariances of the different quadruples of G\F that are produced.  However, that seems a little frustrating, since it says, "if I had made only 240 observations, these would be the uncertainties in my answer."  But with a lot more than 240 actually available, why not make a more reliable estimate with G and F that much larger?
>
> Roger Stafford

Even easier than this would be to look up a very basic textbook on
engineering statistics and obtain the algorithm from that.

Subject: Covariance matrix from a single regression

From: Peter Perkins

Date: 14 Jun, 2010 13:41:41

Message: 5 of 6

On 6/14/2010 12:00 AM, Firat wrote:
> Hi all,
> I am trying to fit a linear model to a set of data. F=G*K is my
> equation, where I know G and F data and try to estimate K. I have 240x1
> F matrix, a G matrix of size 240x4 and K matrix should be 4x1. I can fit
> a linear model to that data and get the K matrix. My question is: Is
> there a way to get the covariance matrix of that single model ?

Assuming you are willing to make all the usual assumptions of multiple
linear regression, I believe that LSCOV is what you are looking for.

Subject: Covariance matrix from a single regression

From: Firat

Date: 14 Jun, 2010 14:23:04

Message: 6 of 6

Thank you very much Peter, that was exactly what I was looking for.

Peter Perkins <Peter.Perkins@MathRemoveThisWorks.com> wrote in message <hv5bil$qvr$1@fred.mathworks.com>...
> On 6/14/2010 12:00 AM, Firat wrote:
> > Hi all,
> > I am trying to fit a linear model to a set of data. F=G*K is my
> > equation, where I know G and F data and try to estimate K. I have 240x1
> > F matrix, a G matrix of size 240x4 and K matrix should be 4x1. I can fit
> > a linear model to that data and get the K matrix. My question is: Is
> > there a way to get the covariance matrix of that single model ?
>
> Assuming you are willing to make all the usual assumptions of multiple
> linear regression, I believe that LSCOV is what you are looking for.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us