Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
less parameter precison using lsqnonlin's jacobian

Subject: less parameter precison using lsqnonlin's jacobian

From: Aidy

Date: 16 Nov, 2011 19:56:28

Message: 1 of 7

hi everyone,

I'd like some advice if possible.
I have a working solution for a nonlinear least squares problem that is solved using lsqnonlin.

When I solve for the unknown parameters I also solve for their corresponding precisions using the following :

%===============================
v=residuals from nonlinear least squares;
J=jacobian;

[rows,cols]=size(J);

% A posteriori variance factor
 Sigma_o = sqrt((v'*v) / (rows-cols));
 
% % Unscaled Covariance Matrix (Inverse of the Normal Equation matrix)
 Q_xx = inv(J'*J);

% Precision measures ,i.e. the standard deviation of each parameter
 diagonals_of_Q_xx = diag(Q_xx);
Precision_of_solved_parameters = Sigma_o .* sqrt(diagonals_of_Q_xx);
%================================

This is the well known method for getting precision(standard deviation) of the solved parameters from least squares.

I solve for the parameters and their corresponding precision using 2 methods: I have my non-linear process which minimizes the objective function without using matlab's optmization toolbox. I specify my own jacobian solution here etc.

My second solution uses lsqnonlin, and I dont specify the Jocobian, instead I estimate it using matlab's finite difference option.

The precision I get from both solutions are different, since the Jacobians from both are numerically different. However, the solved parameters are almost the same for both solutions, so I know there are no inherent errors in the coding. The precisions from using lsqnonlin are larger than that from my code( the first method).

Can anyone help me out on possible reason why this is so?

cheers
aiden

Subject: less parameter precison using lsqnonlin's jacobian

From: Torsten

Date: 17 Nov, 2011 07:27:52

Message: 2 of 7

On 16 Nov., 20:56, "Aidy " <aidenj...@gmail.com> wrote:
> hi everyone,
>
> I'd like some advice if possible.
> I have a working solution for a nonlinear least squares problem that is solved using lsqnonlin.
>
> When I solve for the unknown parameters I also solve for their corresponding precisions using the following :
>
> %===============================
> v=residuals from nonlinear least squares;
> J=jacobian;
>
> [rows,cols]=size(J);
>
> % A posteriori variance factor
>  Sigma_o = sqrt((v'*v) / (rows-cols));
>
> % % Unscaled Covariance Matrix (Inverse of the Normal Equation matrix)
>  Q_xx  = inv(J'*J);
>
> %  Precision measures ,i.e. the standard deviation of each parameter
>  diagonals_of_Q_xx = diag(Q_xx);
> Precision_of_solved_parameters = Sigma_o .* sqrt(diagonals_of_Q_xx);
> %================================
>
> This is the well known method for getting precision(standard deviation) of the solved parameters from least squares.
>
> I solve for the parameters and their corresponding precision using 2 methods: I have my non-linear process which minimizes the objective function without using matlab's optmization toolbox. I specify my own jacobian solution here etc.
>
> My second solution uses lsqnonlin, and I dont specify the Jocobian, instead I estimate it using matlab's finite difference option.
>
> The precision I get from both solutions are different, since the Jacobians from both are numerically different. However, the solved parameters are almost the same for both solutions, so I know there are no inherent errors in the coding. The precisions from using lsqnonlin are larger than that from my code( the first method).
>
> Can anyone help me out on possible reason why this is so?
>
> cheers
> aiden

But already answered your question:
Because the Jacobian matrices obtained from a finite-difference
approximation and
an analytical expression are different.

Best wishes
Torsten.

Subject: less parameter precison using lsqnonlin's jacobian

From: Aidy

Date: 17 Nov, 2011 09:02:15

Message: 3 of 7

Torsten ,

indeed ,there is a different jacobian in the 2 situations. So does this mean that getting the precison of parameters from finte differencing using lsqnonlin is unreliable due to it being larger than a manually coded nonlinear least squares process with a defined Jacobian?

many thanks,
Aiden

Subject: less parameter precison using lsqnonlin's jacobian

From: Torsten

Date: 17 Nov, 2011 11:42:24

Message: 4 of 7

On 17 Nov., 10:02, "Aidy " <aidenj...@gmail.com> wrote:
> Torsten ,
>
> indeed ,there is a different jacobian in the 2 situations. So does this mean that getting the precison of parameters from finte differencing using lsqnonlin is unreliable due to it being larger than a manually coded nonlinear least squares process with a defined Jacobian?
>
> many thanks,
> Aiden

If you have an analytic Jacobian available, you should use it for
speed and precision reasons
- also when working with lsqnonlin.
lsqnonlin offers the possibility of a user-supplied Jacobian by
choosing "Jacobian on" in the options setting.

Best wishes
Torsten.

Subject: less parameter precison using lsqnonlin's jacobian

From: Matt J

Date: 17 Nov, 2011 12:41:10

Message: 5 of 7

"Aidy" wrote in message <ja14hc$3km$1@newscl01ah.mathworks.com>...
>
>
> This is the well known method for getting precision(standard deviation) of the solved parameters from least squares.
================

Just as a side note, you realize that this standard deviation estimate is valid only when minimizing without constraints, correct?

Subject: less parameter precison using lsqnonlin's jacobian

From: Aidy

Date: 17 Nov, 2011 17:59:30

Message: 6 of 7

hello matt,

I am not to familiar with the use of constraints in least squares. I dont think I am using any in my code that does not use lsqnonlin.

However, could it be that unknowingly use it for when I do indeed use lsqnonlin for minimization ? I set some "options" such as :
Algorithm -levenberg marquardt
Tolfun - 1e-6
and a couple more...

let me know if this affects true precision estimate in any way or form...

cheers,
aiden

Subject: less parameter precison using lsqnonlin's jacobian

From: Matt J

Date: 17 Nov, 2011 18:26:30

Message: 7 of 7

"Aidy" wrote in message <ja3i21$33e$1@newscl01ah.mathworks.com>...
> hello matt,
>
> I am not to familiar with the use of constraints in least squares. I dont think I am using any in my code that does not use lsqnonlin.
>
> However, could it be that unknowingly use it for when I do indeed use lsqnonlin for minimization ? I set some "options" such as :
=========

I don't see how you could be applying constraints unknowingly. If you're not using the LB or UB arguments to LSQNONLIN, there will be no constraints.

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us