MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn moreOpportunities for recent engineering grads.

Apply Today
Asked by Steven on 10 Jun 2013

Hi,

I found the coefficients of a simple regression Y = aX1+bX2 using a maximum likelihood optimization. Now I would like to find the t-statistics of coefficient a and b.

The normal regress functions don't allow me to give them as an input though.

Any suggestions how I can do this?

thanks

*No products are associated with this question.*

Answer by Tom Lane on 11 Jun 2013

Edited by Tom Lane on 12 Jun 2013

Accepted answer

If you use regstats to estimate the coefficient standard errors, here's what you get using the Hald data:

load hald s = regstats(heat,ingredients,'linear'); sqrt(diag(s.covb * (8/13)))' % 8/13 converts unbiased variance to mle

If you have another estimator that is the mle for some probability model, you could compute the second derivative of the log likelihood function and use that to estimate the standard errors. Here's an example using the normal distribution so it gives roughly the same answer as above:

loglik = @(b) sum(log(normpdf(heat,b(1)+ingredients*b(2:end),sqrt(8/13)*sqrt(s.mse)))); H = zeros(5); b = s.beta; db = 0.001; for j=1:5, ej = j==(1:5)'; H(j,j) = (loglik(b+ej*db)-2*loglik(b)+loglik(b-ej*db))/db^2; for k=j+1:5 ek = k==(1:5)'; H(j,k) = ( loglik(b+ej*db+ek*db) + loglik(b) ... - loglik(b+ej*db) - loglik(b+ek*db))/db^2; H(k,j) = H(j,k); end end sqrt(diag(inv(-H)))'

Answer by Tom Lane on 11 Jun 2013

The t statistic is the ratio of the estimate to its standard error. If you can determine the standard error, you can take this ratio yourself.

However, least squares is the maximum likelihood method for a regression if the residuals are normally distributed. In that case you can let regress (or regstats or LinearModel) compute the coefficients and t statistics for you. If you have some other estimation method that is not least squares, you would need to determine the standard errors of those estimators.

Steven on 11 Jun 2013

thanks Tom, that's exactly where I'm struggling. I don't know how to calculate the standard error of the estimator. I don't want to use OLS because it gives different coefficients than my maximum likelihood. So I want to calculate the standard error of my actual estimators.

Maybe some more context, I used a Kalman Filter to estimate X1 and X2 and then the maximum likelihood to find the optimal a and b. I now want to find the t-stat of this a and b.

## 0 Comments