Thread Subject: Re: Give me a Regression Problem

On Jul 19, 9:22 am, baldrick <philbrier...@hotmail.com> wrote:
> On Jul 19, 9:11 am, Greg Heath <he...@alumni.brown.edu> wrote:
> > On Jul 18, 1:28 pm, paulvbi...@gmail.com wrote:
> > -------SNIP
>
> > > I just reran the SVD PNN with 3 and 6 missing and with Zaknich
> > > weighting active and get R2=0.9129 which is close to the answer with
> > > all the variables of R2=0.9217 (Zaknich weighting active) which might
> > > just might indicate maybe all the variables should be included
>
> > I wouldn't say that (See below).
>
> > >but
> > > this is not clear to me. Likely there are some subtle second order
> > > interaction effects here, minor for sure, but a little influence
>
> > Since my quadratic models obtained R^2 ~ 0.8, I think the effects
> > are higher order than 2nd.
>
> > > but as Greg says one may not be able to "prove" this also
>
> > I think the best you can do is reject or fail to reject a null
> > hypothesis based on bootstrapping and the probability
> > distribution for a separability measure.
>
> > However, as a practical engineer, I can live with
> > comparing overlaps of mean +/- 2*stdv confidence
> > intervals for MSE estimates obtained from 10-fold XVAL.

I guess I should look up the F-test and see how bad I am.

> Just been doing a bit more messing with the concrete data and it
> appears that drying concrete has a bit of a half life. If you take the
> natural logarithm of the age and use this then you should be able to
> get an r^2 of 0.825 using linear regression. Should save you a few
> neurons.

Nice !

1. The y-to-x8 correlation coefficient increases from 0.33 to 0.55
   and becomes the maximum.
2. The average magnitude of the xi-to-x8 (i=1:7) correlation
coefficients changes from the minimum value of 0.13 to the
lower value of 0.06

3. Linear regression Backward Elimination kept all variables resulting
in
    R^2(adjusted) = 0.817. However, the pvalue for x5 was 0.065 ...
4. Linear regression Forward Selection removed x5 (pvalue = 0.43) in
the selection sequence [ 1 2 3 8 5 4 (5) 7 6] = [ 1 2 3 8
4 7 6] resulting
   in R^2 = 0.816 for p = 7. The final pvalue for x5 was 0.065.

5. Quadratic regression with all p = 44 variables resulted in R^2 =
0.882.
6. Quadratic regression Backward Elimination kept all original
variables
   but removed 7 higher order terms: p = 37, R^2= 0.881.
7. Quadratic regression Forward Selection selected 37 variables
(including
   the original 8). Then removed 4 higher order terms: p = 33, R^2 =
0.880.
8. Quadratic regression starting with the original 8 variables, added
24
   higher order terms then removed 2 of them. p = 30, R^2 = 0.878.
9. Quadratic regression starting with the Paul's 6 variables, added
26
   then removed 2 to obtain the same results as above. x3 was the 1st
   variable added and x6 was the 6th.
10. Finally, target values were standardized. Plotting errors t-y vs
t,
    10 - 12 points with abs(t-y) > ~1 are conspicuous ... Paul's
outliers?

... tempted to try cubic ...

... nah

Hmm ... how many neurons can be saved using log10(age)?

Hope this helps.

Greg