Clear Filters
Clear Filters

possible to reduce the relative difference for CheckGradients in optimization (fmincon)?

6 views (last 30 days)
When double-checking my Jacobian using CheckGradients, I have a relative maximum difference of, crudely, 4e-6,
and my entries of the Jacobian are in the ballpark 1e-1.
I do not expect that Finite Difference can calculate the first six digits after the decimal correctly as my problem has large second derivatives.
That said, can I reduce the relative difference in CheckGradients from 1e-6 to a different value?

Accepted Answer

Matt J
Matt J on 26 Oct 2022
Edited: Matt J on 27 Oct 2022
This FEX download allows you to obtain the results of the finite differencing, as well as to decide where (at what x) you want it computed,
Occasionally also, I find it to be more accurate and reliable as a checking tool than CheckGradient.
In any case, because it gives you the computed FD gradient, you can use this tool to compare both the finite difference and analytical gradients according to whatever error criterion that you wish.

More Answers (1)

Torsten
Torsten on 26 Oct 2022
Edited: Torsten on 26 Oct 2022
Why not just turning off the CheckGradient feature if you are sure your derivatives are correct within a certain error tolerance (which might be larger than 1e-6) ?
  5 Comments
Torsten
Torsten on 27 Oct 2022
To me, the fixed value of 1e-6 does not make sense at all. Say, the entries were in the ballpark 1e-6 -- gradient check would work even if the first relevant digit is wrong. On the other hand, if the entries were 1e-1, gradient check fails although the first 4 or 5 digits are correct.
I don't think that 1e-6 is an absolute, but a relative error in the calculation. Thus
|| gradient_MATLAB - gradient_USER || <= eps * ||gradient_MATLAB||
or something similar is checked for
||gradient_MATLAB|| > 1
and
|| gradient_MATLAB - gradient_USER|| <= eps
only for
||gradient_MATLAB|| <= 1
SA-W
SA-W on 17 Jan 2023
Let me asked one more question regarding your last comment (if you still get notifications about that chat. If not, I will create a new chat).
What you described in your last comment is exactly what is also stated in the documentation.
Example: gradient_USER(1) = 1e-7, gradient_MATLAB(1)=2e-7
|| gradient_MATLAB(1) - gradient_USER(1) || = 1e-7 < 1e-6
This would mean that GradientCheck works, although gradient_USER(1) is, clearly, wrong. Is that true?

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!