Problematic SE with fminunc

5 views (last 30 days)
Deepiks
Deepiks on 8 Jun 2014
Commented: John D'Errico on 9 Jun 2014
Hi I am trying to estimate a discrete choice mixed (logit) model.
The mle (via maximum simulated lik) converges but my standard errors are imaginary. (with fewer simulations I get NaN sometimes).
I use: ihess=inv(hessian); stderr=sqrt(diag(ihess));
The hessian has some columns of zeroes and also negative numbers.
My questions are: 1. What could be causing the imaginary errors? 2. Is there some fix?
  1 Comment
John D'Errico
John D'Errico on 8 Jun 2014
Edited: John D'Errico on 8 Jun 2014
Sorry. I was deleting so many spams, that when I looked at your question because it was of interest to me, I deleted it by accident, out of habit.

Sign in to comment.

Accepted Answer

John D'Errico
John D'Errico on 8 Jun 2014
If the hessian has columns of zeros, then it is only semi-definite. NOT positive definite. So it is not invertible.
Negative numbers in a Hessian are NOT a problem, as long as the MATRIX as a whole is positive definite (think positive eigenvalues.) You don't have that.
There are linear combinations of your variables that yield the same MLE function value, at least a one-dimensional subspace of your parameters has that property.
The EIG or SVD decomposition of your hessian will tell you what that subspace is.
  3 Comments
Deepiks
Deepiks on 8 Jun 2014
Thanks, John.
Can I also give up on trying to get hessian and just bootstrap?
John D'Errico
John D'Errico on 9 Jun 2014
Matt is probably correct, that those zeros are not truly zeros, but tiny enough to display as zero compared to the other elements of the array. This would be a common thing for a MLE problem.
A bootstrap would certainly be a common approach. However, if the hessian is numerically singular, it tells me that any solution in that nullspace will be as good as any other. So if you ran the process over and over again with different starting values, then ANY point from that infinite family of solutions may result. This is what the singularity tells you.
You can get this same information from the Hessian that you already have. The eigenvectors that correspond to the zero/near zero eigenvalues define that subspace for you. And the non-zero eigenvalues can be used to define the spread outside of the nullspace.
Regardless, I don't know that any of this will be that useful, if you are looking for parameter variances.

Sign in to comment.

More Answers (0)

Categories

Find more on Linear Algebra in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!