Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
norm? or R-squared?

Subject: norm? or R-squared?

From: Maurice

Date: 24 Dec, 2003 08:36:17

Message: 1 of 14

Hi,

I've noticed that MATLAB is keen on using the "norm of residuals" as
a measure of how well data fits a polynomial. But I (and more
importantly my boss...) have learned to use the R-squared value as a
measure of "goodness of fit".

While the R-squared value indicates a better fit as it's value
approaches 1, the norm(V,2) indicates a better fit as its value
approaches 0.

Is there a relation between the two? For instance can I say that
R-squared=1-norm(V,2)?

Anyway, is there a function that lets me calculate the R-squared
value for a given set of data and residuals?

Subject: norm? or R-squared?

From: John D'Errico

Date: 24 Dec, 2003 10:30:54

Message: 2 of 14

In article <eecd15d.-1@webx.raydaftYaTP>,
 Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:

> Hi,
>
> I've noticed that MATLAB is keen on using the "norm of residuals" as
> a measure of how well data fits a polynomial. But I (and more
> importantly my boss...) have learned to use the R-squared value as a
> measure of "goodness of fit".
>
> While the R-squared value indicates a better fit as it's value
> approaches 1, the norm(V,2) indicates a better fit as its value
> approaches 0.
>
> Is there a relation between the two? For instance can I say that
> R-squared=1-norm(V,2)?
>
> Anyway, is there a function that lets me calculate the R-squared
> value for a given set of data and residuals?

Interesting. I tend to ignore R^2. It has dangers
that many do not realize. For example, R^2 is
invalid when there is no constant term in your
model. (One can modify R^2 for models with no
constant term, but it is inconvenient.)

Suppose I told you a model for some data had an R^2
of 0.975. Is the model adequate for my purposes?
What if the R^2 is 0.999?

Your response should be that you have no idea.
My data may well be very accurate, so an R^2 of
only 0.975 may well be a relatively poor fit.
I needed to tell you what my goals were in fitting
this specific model before you have any idea of
what the fit quality should be. I also needed to
tell you how good the data is, etc. Since R^2 is
a dimensionless number, you really cannot say
anything.

I'll argue that only by looking at the residuals,
plotting the curve fit, etc., can you really
assess how good a fit is. Mono-numerosis, i.e.,
the use of a single statistic to gauge the quality
of a model, in any fitting scheme is dangerous.

Having climbed on top of that soapbox, I'll step
down for a moment. What is the definition of R^2?
How would you compute it? Don't tell me that you
have no idea. If you do not know then you have
no business in using R^2 in the first place.
The question you ask above tells me you do not
know. UNDERSTAND YOUR TOOLS! Otherwise it is all
a complete waste of time. So, I repeat. What is
R^2?

   R^2 = 1 - ss_regression/ss_total

The idea is you divide the regression sum of
squares by the total sum of squares, then subtract
it from 1. What is the regression sum of squares?
Its the square of the norm of your residuals.
How about the total sum of squares? Does it change
due to your regression model? No. So, minimizing
the norm of the residuals is the mathematical
equivalent of maximizing R^2. Except...

Practically, you can see subtractive cancellation
problems in computing R^2. This can be crucially
important in a nonlinear optimization near the
solution. (If your model is nonlinear. If it is
a polynomial model, then no iterative scheme is
required.)

The point of all this is to suggest that you
should not look at any single measure of fit.
Learn data analysis and statistical modeling
so that you know when and how to apply the proper
measures of fit. Learn to understand your model.
Learn about your data and its sources of error.
If you do all this, then R^2 will seem trivial
to you too.

You never know. You might even end up impressing
that boss of yours one day.

HTH,
John D'Errico


--
There are no questions "?" about my real address.

The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: norm? or R-squared?

From: us

Date: 24 Dec, 2003 10:59:23

Message: 3 of 14

John D'Errico:
<SNIP some lengthy educational piece to a newcomer (?) ... ending
with:

> Learn about your data and its sources of error. If you do all this,
then R^2 will seem trivial to you too.

very well said, john. couldn't agree more. it's like hearing myself
saying to my students: if you don't see what's going on in the RAW
data - do not bother about statistics, let alone any second order
derivative of it!

us

Subject: norm? or R-squared?

From: Steve Lord

Date: 24 Dec, 2003 11:37:57

Message: 4 of 14

Maurice wrote:
>
>
> Hi,
>
> I've noticed that MATLAB is keen on using the "norm of residuals"
> as
> a measure of how well data fits a polynomial. But I (and more
> importantly my boss...) have learned to use the R-squared value as
> a
> measure of "goodness of fit".
>
> While the R-squared value indicates a better fit as it's value
> approaches 1, the norm(V,2) indicates a better fit as its value
> approaches 0.
>
> Is there a relation between the two? For instance can I say that
> R-squared=1-norm(V,2)?
>
> Anyway, is there a function that lets me calculate the R-squared
> value for a given set of data and residuals?

Assuming you've read John's posting about the dangers of relying on
R^2 and still want to use it, then yes; use CORRCOEF to compute the
correlation coefficient between your experimental data and the data
that the model you've fit.

--
Steve Lord
slord@mathworks.com

Subject: norm? or R-squared?

From: Maurice

Date: 25 Dec, 2003 04:15:34

Message: 5 of 14

Steve Lord wrote:
>
>
> Maurice wrote:
>>
>>
[snip]
>
> Assuming you've read John's posting about the dangers of relying on
> R^2 and still want to use it, then yes; use CORRCOEF to compute the
> correlation coefficient between your experimental data and the data
> that the model you've fit.
>
> --
> Steve Lord
> slord@mathworks.com

Thanks. That's what I've been looking for. I haven't got MATLAB
available here, is that part of the Statistics toolbox (cause I don't
have that)?

Subject: norm? or R-squared?

From: Maurice

Date: 25 Dec, 2003 05:16:18

Message: 6 of 14

John D'Errico wrote:
>
>
> In article <eecd15d.-1@webx.raydaftYaTP>,
> Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:
>
>> Hi,
>>
[snip]

Thanks for the disertation. I have to acknowledge that the norm of
the residuals may be a better way to evaluate a fit, however I have
little experience with it and don't know whether a norm of 0.018
would be a good fit. And what about 0.18?
Anyway there are industries where procedures are not easily changed.
And when everybody around the world you work with insists on using
R^2 you have little chance of changing that on short notice. I grant
you that the norm() function is interesting and I think I should
study it more closely so that I can try and talk my boss and
colleagues into changing their way. But, I don't have the illusion
that will happen overnight. And until it does I'm bound to using R^2
with all the dangers that come with it.

I *do* know what R^2 means and in school I was taught to use this
(statistics class). I never heard of the norm of residuals before a
few days ago. I also know how to calculate R^2. But, I think I should
not write a new function if it is already included in MATLAB.
Re-inventing the wheel is not wise. Anyway, what I'm trying to do is
fit a set of experimental data to a linear function. For example:

x=[4 8 16 24]
y=[0.15 0.3 0.45 0.7]
[P,S]=polyfit(x,y,1)

% S.normr turns out to be 0.046029. Is that % a good fit?

% Then I do:

yh=polyval(P,x)
residuals=y-yh
norm(residuals) %should be same as S.normr
%and it is

% Now, what I understand how R^2 works is
% as follows:

rss=sum((yh-mean(y)).^2)
tss=sum((y -mean(y)).^2)

% My definition of R^2:

Rsquared=rss/tss

% Rsquared is 0.98716 which I would judge
% as a reasonably good fit.

% John's definition of R^2

Rsquared=1-(rss/tss)

Rsquared in this case is 0.01284 which does not make sense if you
look at the plot

plot(x,y,'o',x,yh)

Looking at the plot I'd expect a good fit.

Okay, the above is from memory so maybe I made a mistake and in that
case I'd appreciate it if it could be pointed out to me.

The whole point of my question was to find out if there is a standard
function already in MATLAB that would do the job for me (I don't have
the statistics toolbox). I understand corrcoef() might do the job.

Thanks for your time and input.
Happy holidays

Subject: norm? or R-squared?

From: John D'Errico

Date: 25 Dec, 2003 09:38:28

Message: 7 of 14

In article <eecd15d.4@webx.raydaftYaTP>,
 Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:

> Thanks for the disertation. I have to acknowledge that the norm of
> the residuals may be a better way to evaluate a fit, however I have
> little experience with it and don't know whether a norm of 0.018
> would be a good fit. And what about 0.18?


Exactly my point. (I'll pretend that you have transformed
the residual norm to be a standard deviation of the errors,
my own preference.)

If you know nothing about your data to know whether
your residual error is of the size 0.18 or .018, then
you have little business fitting this data.

If you know nothing about your data, then you also have
no idea what level of R^2 is acceptable either. Don't tell
me that an R^2 of 0.98716 is good enough for you. I can
easily give you data where that R^2 is equivalent to a
ridiculously poor fit, or one where it corresponds to an
equally ridiculous over-fit.


> Anyway there are industries where procedures are not easily changed.

I've seen corporate inertia. I've also seen that people
can & will learn from somebody who has something worth
teaching.


> And when everybody around the world you work with insists on using
> R^2 you have little chance of changing that on short notice. I grant

Teach them by example. You can still include the value
of R^2 on your plots. That does not mean you need to
use it as your guide to the universe.


> you that the norm() function is interesting and I think I should
> study it more closely so that I can try and talk my boss and
> colleagues into changing their way. But, I don't have the illusion
> that will happen overnight. And until it does I'm bound to using R^2
> with all the dangers that come with it.

No. You are bound to report R^2.

Learn to use it in conjunction with other measures
of fit.

More importantly, learn to use your eyes. Learn to
look at residual plots. Never let any single number
do your thinking for you.


> I *do* know what R^2 means and in school I was taught to use this
> (statistics class). I never heard of the norm of residuals before a

Forget about the residual norm. Transform it into the
standard deviation of the residuals. Now it makes sense
to you as a number with the proper units and magnitude.

How does it compare to the magnitude of the error you
expect to see in this data?


> % Now, what I understand how R^2 works is
> % as follows:
>
> rss=sum((yh-mean(y)).^2)
> tss=sum((y -mean(y)).^2)
>
> % My definition of R^2:
>
> Rsquared=rss/tss
>
> % Rsquared is 0.98716 which I would judge
> % as a reasonably good fit.
>
> % John's definition of R^2
>
> Rsquared=1-(rss/tss)

Let us be careful here. If you define the residual sum
of squares as the sum of squares of the residuals, then
the formula above does make sense. As the sum of squares
of residuals goes to zero, you want R^2 to approach 1.

Don't just learn the formulas. Learn where they come
from and what they mean.


> The whole point of my question was to find out if there is a standard
> function already in MATLAB that would do the job for me (I don't have
> the statistics toolbox). I understand corrcoef() might do the job.

There is no explicit R^2 function in matlab. It takes
very little to write one however. The power of matlab
lies in its extensibility. As you add tools like this
of your own, customized to your own class of problems,
you will quickly build up a very nice library of tools.

Allow your co-workers to share in your wealth by using
these same tools. You might be pleased in where it all
leads over the course of a career. (Spoken by somebody
who has spent his entire career providing libraries of
tools to a major corporation.)

John


--
There are no questions "?" about my real address.

The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: norm? or R-squared?

From: Maurice

Date: 26 Dec, 2003 02:35:52

Message: 8 of 14

John D'Errico wrote:
>
>
> In article <eecd15d.4@webx.raydaftYaTP>,
> Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:
>
>> Thanks for the disertation. I have to acknowledge that the norm
> of
>> the residuals may be a better way to evaluate a fit, however I
> have
>> little experience with it and don't know whether a norm of
0.018
>> would be a good fit. And what about 0.18?
>
>
> Exactly my point. (I'll pretend that you have transformed
> the residual norm to be a standard deviation of the errors,
> my own preference.)
>
> If you know nothing about your data to know whether
> your residual error is of the size 0.18 or .018, then
> you have little business fitting this data.
>
> If you know nothing about your data, then you also have
> no idea what level of R^2 is acceptable either. Don't tell
> me that an R^2 of 0.98716 is good enough for you. I can
> easily give you data where that R^2 is equivalent to a
> ridiculously poor fit, or one where it corresponds to an
> equally ridiculous over-fit.

I'm always willing to learn. Please share those data with me.

Subject: norm? or R-squared?

From: nztideman@yahoo.co.nz (Derek Goring)

Date: 26 Dec, 2003 01:34:23

Message: 9 of 14

snip a lot of pedagogic fussiness about r^2 (on Christmas Day too :-)

For a practising data analyst, r^2 is simply the proportion of the
variance of y that can be explained by x, so for example if I regress
atmospheric pressure against storm surge and find r^2 = 0.700 (which
is typical), that means that local variations in pressure explain 70%
of the variance in storm surge, and that I need to go find what is
causing the other 30% (usually wind effects).

Derek

Subject: norm? or R-squared?

From: John D'Errico

Date: 26 Dec, 2003 18:29:23

Message: 10 of 14

In article <eecd15d.6@webx.raydaftYaTP>,
 Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:

> John D'Errico wrote:
> >
> >
> > In article <eecd15d.4@webx.raydaftYaTP>,
> > Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:
> >
> >> Thanks for the disertation. I have to acknowledge that the norm
> > of
> >> the residuals may be a better way to evaluate a fit, however I
> > have
> >> little experience with it and don't know whether a norm of
> 0.018
> >> would be a good fit. And what about 0.18?
> >
> >
> > Exactly my point. (I'll pretend that you have transformed
> > the residual norm to be a standard deviation of the errors,
> > my own preference.)
> >
> > If you know nothing about your data to know whether
> > your residual error is of the size 0.18 or .018, then
> > you have little business fitting this data.
> >
> > If you know nothing about your data, then you also have
> > no idea what level of R^2 is acceptable either. Don't tell
> > me that an R^2 of 0.98716 is good enough for you. I can
> > easily give you data where that R^2 is equivalent to a
> > ridiculously poor fit, or one where it corresponds to an
> > equally ridiculous over-fit.
>
> I'm always willing to learn. Please share those data with me.

Acceptability of fit is a function of your requirements
for a model for the data. What do you intend/need to do
with the model? How large is the noise?

x = rand(25,1);
y = exp(x/2)+randn(size(x))/65;
p=polyfit(x,y,1);

This will yield an R^2 of roughly 0.987 from my quick
test. Is it a poor model? It depends upon how much you
need to fit the subtle curvature in that data. This
case shows that the model chosen (a linear one) may
be adequate, or may not. It depends on your data and
your needs.

How about this data?

x = rand(25,1);
y = x + ((x-.5).^2)*.46;
p=polyfit(x,y,1);

A linear model will also yield R^2 about 0.987. (It
does depend a bit on the random sample chosen.)

Plot this data with the linear fit on top. The
quadratic lack of fit here is quite obvious, and
would seem to indicate a linear model is clearly
inadequate.

If you want to generate a ridiculus overfit, just
do something as simple as:

x = rand(20,1);
y = x+randn(size(x))*.1;

Now fit it with a high order polynomial. I won't bother
wasting my time. I claim that a high enough order
polynomial (maybe order 15 to 17?) will generate an
arbitrarily "good" R^2. Is this a good model? Plot
the resulting polynomial. What does this model do
between the data points?

Can you honestly say that R^2 tells enough about any
of these models to decide if the fit is adequate?

John


--
There are no questions "?" about my real address.

The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: norm? or R-squared?

From: John D'Errico

Date: 26 Dec, 2003 19:02:44

Message: 11 of 14

In article <eecd15d.4@webx.raydaftYaTP>,
 Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:

>
> Anyway there are industries where procedures are not easily changed.
> And when everybody around the world you work with insists on using
> R^2 you have little chance of changing that on short notice. I grant
> you that the norm() function is interesting and I think I should
> study it more closely so that I can try and talk my boss and
> colleagues into changing their way. But, I don't have the illusion
> that will happen overnight. And until it does I'm bound to using R^2
> with all the dangers that come with it.

I'm reminded of one time, 1979 or so, when I got a
severe dressing down by my boss. I had done nothing
more than suggest the use of a spline model instead
of a nonlinear regrssion model for a certain class
of data. I had suggested one to our client, and my
boss had overheard me.

We did things in a certain way in those days. Those
ways did not included spline models. I'll say only
that spline models are now more prevalent in that
same large corporation. This is largely because of
my perseverence in introducing them, teaching their
value, and providing the software to use them.
(In 1979, I had only used splines for a year or
so. So I learned what I needed along the way.)

Corporate cultures can be changed. It took some
years in this case. It helps if you are willing to
invest the time. Learn what you need. Teach others.

If you decide you cannot make a difference, then
you have no chance.

John


--
There are no questions "?" about my real address.

The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: norm? or R-squared?

From: John D'Errico

Date: 1 Jan, 2004 01:45:37

Message: 12 of 14

In article <d94b83b8.0312260134.6d355aa5@posting.google.com>,
 nztideman@yahoo.co.nz (Derek Goring) wrote:

> snip a lot of pedagogic fussiness about r^2 (on Christmas Day too :-)
>
> For a practising data analyst, r^2 is simply the proportion of the
> variance of y that can be explained by x, so for example if I regress
> atmospheric pressure against storm surge and find r^2 = 0.700 (which
> is typical), that means that local variations in pressure explain 70%
> of the variance in storm surge, and that I need to go find what is
> causing the other 30% (usually wind effects).
>
> Derek

As another practising data analyst...

To take your example, if you choose an arbitrary
polynomial model for some data, and the R^2 just
happens to be 0.7, then can you really conclude
anything beyond that fact?

Have I told you what component of the residual
error was lack of fit? Did you forget to look?

What have you learned here as a practising data
analyst? Is the model adequate? Can you make that
decision knowing nothing more than R^2?

John


--
There are no questions "?" about my real address.

The best material model of a cat is another, or
preferably the same, cat.
A. Rosenblueth, Philosophy of Science, 1945

Subject: norm? or R-squared?

From: Shiguo

Date: 25 May, 2012 03:28:32

Message: 13 of 14

Nice input John. I totally agree with you. I've read many of your posts and downloaded your files.

John D'Errico <?derrico@flare.n?e?t> wrote in message <?derrico-440D83.10305324122003@news.newsguy.com>...
> In article <eecd15d.-1@webx.raydaftYaTP>,
> Maurice <p12tpe1.nospam@yahoo.co.uk> wrote:
>
> > Hi,
> >
> > I've noticed that MATLAB is keen on using the "norm of residuals" as
> > a measure of how well data fits a polynomial. But I (and more
> > importantly my boss...) have learned to use the R-squared value as a
> > measure of "goodness of fit".
> >
> > While the R-squared value indicates a better fit as it's value
> > approaches 1, the norm(V,2) indicates a better fit as its value
> > approaches 0.
> >
> > Is there a relation between the two? For instance can I say that
> > R-squared=1-norm(V,2)?
> >
> > Anyway, is there a function that lets me calculate the R-squared
> > value for a given set of data and residuals?
>
> Interesting. I tend to ignore R^2. It has dangers
> that many do not realize. For example, R^2 is
> invalid when there is no constant term in your
> model. (One can modify R^2 for models with no
> constant term, but it is inconvenient.)
>
> Suppose I told you a model for some data had an R^2
> of 0.975. Is the model adequate for my purposes?
> What if the R^2 is 0.999?
>
> Your response should be that you have no idea.
> My data may well be very accurate, so an R^2 of
> only 0.975 may well be a relatively poor fit.
> I needed to tell you what my goals were in fitting
> this specific model before you have any idea of
> what the fit quality should be. I also needed to
> tell you how good the data is, etc. Since R^2 is
> a dimensionless number, you really cannot say
> anything.
>
> I'll argue that only by looking at the residuals,
> plotting the curve fit, etc., can you really
> assess how good a fit is. Mono-numerosis, i.e.,
> the use of a single statistic to gauge the quality
> of a model, in any fitting scheme is dangerous.
>
> Having climbed on top of that soapbox, I'll step
> down for a moment. What is the definition of R^2?
> How would you compute it? Don't tell me that you
> have no idea. If you do not know then you have
> no business in using R^2 in the first place.
> The question you ask above tells me you do not
> know. UNDERSTAND YOUR TOOLS! Otherwise it is all
> a complete waste of time. So, I repeat. What is
> R^2?
>
> R^2 = 1 - ss_regression/ss_total
>
> The idea is you divide the regression sum of
> squares by the total sum of squares, then subtract
> it from 1. What is the regression sum of squares?
> Its the square of the norm of your residuals.
> How about the total sum of squares? Does it change
> due to your regression model? No. So, minimizing
> the norm of the residuals is the mathematical
> equivalent of maximizing R^2. Except...
>
> Practically, you can see subtractive cancellation
> problems in computing R^2. This can be crucially
> important in a nonlinear optimization near the
> solution. (If your model is nonlinear. If it is
> a polynomial model, then no iterative scheme is
> required.)
>
> The point of all this is to suggest that you
> should not look at any single measure of fit.
> Learn data analysis and statistical modeling
> so that you know when and how to apply the proper
> measures of fit. Learn to understand your model.
> Learn about your data and its sources of error.
> If you do all this, then R^2 will seem trivial
> to you too.
>
> You never know. You might even end up impressing
> that boss of yours one day.
>
> HTH,
> John D'Errico
>
>
> --
> There are no questions "?" about my real address.
>
> The best material model of a cat is another, or
> preferably the same, cat.
> A. Rosenblueth, Philosophy of Science, 1945

Subject: norm? or R-squared?

From: Greg Heath

Date: 25 May, 2012 21:27:00

Message: 14 of 14

On May 24, 11:28 pm, "Shiguo " <jiangshi...@gmail.com> wrote:
> Nice input John. I totally agree with you. I've read many of your posts and downloaded your files.
>
> John D'Errico <?derr...@flare.n?e?t> wrote in message <?derrico-440D83.10305324122...@news.newsguy.com>...
> > In article <eecd15d...@webx.raydaftYaTP>,
> > Maurice <p12tpe1.nos...@yahoo.co.uk> wrote:
>
> > > Hi,
>
> > > I've noticed that MATLAB is keen on using the "norm of residuals" as
> > > a measure of how well data fits a polynomial. But I (and more
> > > importantly my boss...) have learned to use the R-squared value as a
> > > measure of "goodness of fit".
>
> > > While the R-squared value indicates a better fit as it's value
> > > approaches 1, the norm(V,2) indicates a better fit as its value
> > > approaches 0.
>
> > > Is there a relation between the two? For instance can I say that
> > > R-squared=1-norm(V,2)?
>
> > > Anyway, is there a function that lets me calculate the R-squared
> > > value for a given set of data and residuals?
>
> > Interesting. I tend to ignore R^2. It has dangers
> > that many do not realize. For example, R^2 is
> > invalid when there is no constant term in your
> > model. (One can modify R^2 for models with no
> > constant term, but it is inconvenient.)
>
> > Suppose I told you a model for some data had an R^2
> > of 0.975. Is the model adequate for my purposes?
> > What if the R^2 is 0.999?
>
> > Your response should be that you have no idea.
> > My data may well be very accurate, so an R^2 of
> > only 0.975 may well be a relatively poor fit.
> > I needed to tell you what my goals were in fitting
> > this specific model before you have any idea of
> > what the fit quality should be. I also needed to
> > tell you how good the data is, etc. Since R^2 is
> > a dimensionless number, you really cannot say
> > anything.
>
> > I'll argue that only by looking at the residuals,
> > plotting the curve fit, etc., can you really
> > assess how good a fit is. Mono-numerosis, i.e.,
> > the use of a single statistic to gauge the quality
> > of a model, in any fitting scheme is dangerous.
>
> > Having climbed on top of that soapbox, I'll step
> > down for a moment. What is the definition of R^2?
> > How would you compute it? Don't tell me that you
> > have no idea. If you do not know then you have
> > no business in using R^2 in the first place.
> > The question you ask above tells me you do not
> > know. UNDERSTAND YOUR TOOLS! Otherwise it is all
> > a complete waste of time. So, I repeat. What is
> > R^2?
>
> > R^2 = 1 - ss_regression/ss_total
>
> > The idea is you divide the regression sum of
> > squares by the total sum of squares, then subtract
> > it from 1. What is the regression sum of squares?
> > Its the square of the norm of your residuals.
> > How about the total sum of squares? Does it change
> > due to your regression model? No. So, minimizing
> > the norm of the residuals is the mathematical
> > equivalent of maximizing R^2. Except...
>
> > Practically, you can see subtractive cancellation
> > problems in computing R^2. This can be crucially
> > important in a nonlinear optimization near the
> > solution. (If your model is nonlinear. If it is
> > a polynomial model, then no iterative scheme is
> > required.)
>
> > The point of all this is to suggest that you
> > should not look at any single measure of fit.
> > Learn data analysis and statistical modeling
> > so that you know when and how to apply the proper
> > measures of fit. Learn to understand your model.
> > Learn about your data and its sources of error.
> > If you do all this, then R^2 will seem trivial
> > to you too.
>
> > You never know. You might even end up impressing
> > that boss of yours one day.
>
> > HTH,
> > John D'Errico

I have been designing neural networks for ~40 yrs and have been
contributing to comp.ai.neural-nets and comp.soft-sys.matlab since
~1997. For those last 15 years I have seen every imaginable
perturbation of misuse and abuse associated with designing NNs for
classification and regression.

For example:

1. No regard for the physical and/or mathematical source of the data.
2. No attempt to visualize the data re clustering and/or I/O variable
plots.
3. No consideration of multiple input scales, multiple output scales
and outliers.
4. No consideration of simpler models; e.g.,
    a. Constant (output = mean(target data))
    b. Linear
5. No attempt to simplify the model by reducing the number of hidden
nodes.
6. No attempt to simplify the model by removing insignificant and/or
redundant inputs.
7. No degree-of-freedom adjustments when characterizing the model via
performance on training data.

Quite often the reason is obvious:

"I'm in a hurry to get an answer and hand in the assignment before the
deadline."

However, I get the feeling that some of the blame rests on the
shoulders of teachers and/or advisors.

End of rant.

I often interpret R^2 in terms of the normalized mean-squared error

NMSE = MSE/MSE00 = 1 - R^2

where MSE00 is the mse obtained from the above-mentioned Constant
Model. For nontraining data, the MSE is proportional to the sum-square-
error, SSE, via

MSE = SSE/Neq and MSE00 = SSE00/Neq

where

Neq = N*O

is the total number of scalar training equations (N = number of data
points and O = number of output variables). Consequently, R^2 can
also be interpreted in terms of the normalized sum-squared-error

NSSE = SSE/SSE00 = 1 - R^2.

However, if the training data is used to obtain the measures, the
corresponding optimistic bias should be mitigated via degree-of-
freedom-adjustments. For example,

MSEa = SSE/(Neq-Nw) and MSE00a = SSE00/(Neq-Nw00)

Where Nw represents the loss in degrees-of-freedom by using the same
data to estimate both Nw weights and corresponding model errors. It
should be noted that

1. MSEa --> inf as Nw --> Neq and is undefined for Nw >= Neq

2. Nw00 = O and MSE00a = mean(training data output variances).

The resulting adjustments yield

NMSEa = MSEa/MSE00a = 1 - Ra^2.

I'll let the reader contemplate the direct relationship between R^2
and Ra^2.
I can't find anything interesting to say about it.

Questions:

1. Is R considered to be the multivariate correlation coefficient?
2. If so how is Ra interpreted?

Greg

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us