Clear Filters
Clear Filters

Check Gradient (Jacobian) of objective function -- what is the meaning of absolute difference (1e-6) if component of gradient is less than 1 ?

5 views (last 30 days)
I use lsqnonlin to solve my data-fitting problem and provide the Jacobian, which I verify using CheckGradients option.
As stated here, if a component of the Jacobian is less than 1, gradient check is successful if the absolute difference between the user-shipped Jacobian and Matlabs finite-difference approximation of that component is less than 1e-6.
Example: Jacobian_USER(1,1)=2e-7, Jacobian_MATLAB(1,1)=1e-7.
Here, Jacobian_USER(1,1) is clearly wrong, but Jacobian check would be successful. Is that true?
  11 Comments
SA-W
SA-W on 18 Jan 2023
"You shouldn't assume something is wrong if the result of one computation gives 2e-7, the result of another computation gives 1e-7."
Why should I not be alarmed in that case?
If I know my values are so small (around 1e-7) per nature, a difference between 1e-7 and 2e-7 is significant.
Torsten
Torsten on 18 Jan 2023
Edited: Torsten on 18 Jan 2023
Why should I not be alarmed in that case?
If I know my values are so small (around 1e-7) per nature, a difference between 1e-7 and 2e-7 is significant.
The values you investigate are numerical derivatives (usually of complex functions). Already when choosing the h in approximating the derivative as (f(x+h)-f(x))/h can cause such differences.

Sign in to comment.

Accepted Answer

Matt J
Matt J on 18 Jan 2023
Edited: Matt J on 18 Jan 2023
If I know my values are so small (around 1e-7) per nature, a difference between 1e-7 and 2e-7 is significant.
Not necessarily. The element-wise relative error is large, but remember lsqnonlin uses the Jacobian J in matrix products of the form J*x and J.'*y to compute its updates. If the average value of abs(J(m,n)) is 1000 and the average values of abs(x(n)) or abs(y(m)) is 10, then an absolute error of 1e-7 in any entry J(m,n) would likely contribute negligibly to J*x or J.'*y.
Regardless, if you don't have full confidence in DerivativeCheck, you can implement your own finite difference computation, using for example,
and then do the comparison according to a criterion you prefer.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!