Calculating covariance matrix from Jacobian using lsqcurvefit

Asked by Wolfgang Klassen

Wolfgang Klassen (view profile)

on 4 Jun 2019
Latest activity Commented on by Bram Stegeman

Bram Stegeman (view profile)

on 4 Sep 2019
Accepted Answer by Alan Weiss

Alan Weiss (view profile)

I am trying to calculate the covariance matrix from the residuals vector and the Jacobian matrix, which are optional outputs of the lsqcurvefit function. I keep getting negative values for the diagonal (variance) values, but they should be strictly positive. Does this have something to do with the method that lsqcurvefit uses? I have attached sample data and a sample extPar structure (this usually gets generated other peices of code that call this function). The part of the code that I think is germane is here:
[par,resnorm,R,~,~,~,J] = lsqcurvefit(FID,par0,fitTime,fitData);
pCov = inv(J'*J)*((R'*R)./(numel(fitData) - numel(par)));
The fit function converges to a nice looking solution, am I doing something drastically wrong here?
I am fully aware that the statistics toolbox solves this easily, but unless you give my prof the cash to buy it for me, I'm stuck with this.

David Wilson

David Wilson (view profile)

on 5 Jun 2019
Your analysis looks reasonable. The matrix J'*J is positive definite by definition, and so is its inverse. R'*R will be positive and so is nu = n-m. However numerical ill-conditioning could be the problem. Check the eigenvalues of inv(J'*J), they should be all +ve (or zero). What is the condition # or rcond of J'*J ?
Of course there are square root methods (SVD etc) to do this that avoid the possible ill-conditioning.

Release

R2019a

Answer by Alan Weiss

Alan Weiss (view profile)

on 6 Jun 2019

This old documentation example might be of some use.
Alan Weiss
MATLAB mathematical toolbox documentation

Wolfgang Klassen

Wolfgang Klassen (view profile)

on 6 Jun 2019
This is fantastic! Thank you!
Bram Stegeman

on 4 Sep 2019
Perfect!

Answer by Matt J

on 5 Jun 2019
Edited by Matt J

Matt J (view profile)

on 5 Jun 2019

J'*J is only guaranteed to be positive semi-definite, which means some of the diagonals can be zero in ideal math. However, computers do not perform ideal math. Due to numerical precision limitations, the calculation can have small inaccuracies and give you slighly negative values in diagonals that are supposed to be zero.