Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Are two models significantly different?

Subject: Are two models significantly different?

From: sam

Date: 8 Jun, 2012 07:58:07

Message: 1 of 9

Dear All

Say we have two regression models, one before and one after an overhaul of a system. How would one determine if the two models are significantly different? Maybe via residual analysis?

Any hints please?

Subject: Are two models significantly different?

From: sam

Date: 12 Jun, 2012 06:21:07

Message: 2 of 9


Anybody?

Subject: Are two models significantly different?

From: Star Strider

Date: 12 Jun, 2012 13:49:07

Message: 3 of 9



I haven't needed to compare models in a long time and am in the process of re-learning the statistics behind it. One paper I am finding useful is:

http://orfe.princeton.edu/~jqfan/papers/99/likeratio.pdf

It is available for free.

Subject: Are two models significantly different?

From: sam

Date: 13 Jun, 2012 06:25:09

Message: 4 of 9



Thank you for the paper and efforts, but I believe that the answer is much simpler :)

Thank you!!

Subject: Are two models significantly different?

From: Torsten

Date: 13 Jun, 2012 07:50:01

Message: 5 of 9

On 8 Jun., 09:58, "sam " <smj...@gmail.com> wrote:
> Dear All
>
> Say we have two regression models, one before and one after an overhaul of a system. How would one determine if the two models are significantly different? Maybe via residual analysis?
>
> Any hints please?

Could you give us a hint of what you mean by "two models are
significantly different" ?
I know that analysis of variance can test whether the _mean values_ of
two _data sets_ are significantly different.
Is it that what you mean ?

The sci.stat.math forum seems to be better suited to answer questions
like yours, but the formulation of
your problem should be much more precise.

Best wishes
Torsten.

Subject: Are two models significantly different?

From: Greg Heath

Date: 13 Jun, 2012 14:50:07

Message: 6 of 9

"sam" wrote in message <jqsbaf$db6$1@newscl01ah.mathworks.com>...
> Dear All
>
> Say we have two regression models, one before and one after an overhaul of a system. How would one determine if the two models are significantly different? Maybe via residual analysis?
>
> Any hints please?

StatisticsToolbox/UsersGuide/MultivariateMethods/ProcrustesAnalysis
StatisticsToolbox/Functions/MultivariateMethods/ProcrustesAnalysis

Hope this helps.

Greg

Subject: Are two models significantly different?

From: Star Strider

Date: 13 Jun, 2012 16:44:07

Message: 7 of 9

"sam" wrote in message <jr9bo5$sfl$1@newscl01ah.mathworks.com>...
>
>
> Thank you for the paper and efforts, but I believe that the answer is much simpler :)
>
> Thank you!!


I agree with Torsten — you need to provide more details. When you mentioned 'residual analysis', I thought you were suggesting likelihood ratio analysis, which is generally the way to directly compare different nonlinear regression models with the same data to determine the model that best fits the data.

I suggest you look at:

>> doc NonLinearModel

The 'NonLinearModel class' (Statistics Toolbox) has a number of options and functions that may provide the sort of test statistics you want on both the parameters and the fit.

Subject: Are two models significantly different?

From: sam

Date: 13 Jun, 2012 17:45:08

Message: 8 of 9

Thank you guys for contributions!

Procrustes Analysis looks good idea.

Since there is interest I'll explain:
We would like to evaluate if an overhaul of our system was successful. The problem is that we cannot perform experiments to evaluate it. Hence credible ANOVA of covariance matrices is probably not possible to do. But we have enough of data before and after the overhaul, so we can constructed "before" and "after" models: both X -> y. The reparation should have a positive effect on variable y. We assume that if an overhaul was successful, the residuals of the "after" model in y should be:
(1) smaller as well as
(2) "more similar" to white noise

than in "before" model.

So, how could one evaluate these two things based on before and after models?

Subject: Are two models significantly different?

From: Star Strider

Date: 13 Jun, 2012 22:00:07

Message: 9 of 9


I am absolutely lost. I have no idea what you are doing, what your model is, or what your data look like. I am out of ideas.

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us