MATLAB Answers

Aaronne
0

How does Matlab build-in Quasi-Newton method calculate the gradient?

Asked by Aaronne
on 20 May 2011
Hi there,
I use matlab build-in function 'fminunc' to run some optimization for my image registration problem. I set the options as:
options = optimset('LargeScale', 'off', 'Display', 'iter-detailed', 'TolX', 0.0001, 'TolFun', 0.001, 'GradObj', 'off', 'DerivativeCheck', 'off';
I think it then uses Quasi-Newton (L-BFGS) method to do the optimization.
When the 'GradObj' is 'off', then the 'fminunc' calculate the gradient of the parameters need to be optimzed for me. (I suppose the Matlab use finite difference method to get the gradient)
I also try to provide the gradient by myself using finite difference method as well, then I set 'GradObj' is 'on'.
However, I always got much better results when I use the Matlab's calculation (When 'GradObj' is 'off').
May I ask where can I find the codes the matlab build-in function calculating the gradient please? I wish to compare with my own implementation to find the mistakes I might make.
Thanks very much and have a nice day.
Aaron

  0 Comments

Sign in to comment.

2 Answers

Answer by Andy
on 20 May 2011

I don't have a direct answer for you, but you can dig into it a little bit further by typing
edit fminunc
at the command line. It looks like some internal functions are called, not all of which are editable (perhaps some are MEX functions?). But you can see a couple of them:
edit optimfcnchk
edit sfminbx
Perhaps you can gain some insight from these functions, but it may just be the case that the algorithm is buried in a MEX function and TMW does not intend for users to be able to see it.

  1 Comment

Ree, your "answer" below is not really another answer, but is a comment on this answer. It would be better for you to use the "comment" feature. The answers could change order over time, and it will be difficult to tell what your comment is responding to.
As for the finitedifferences function, it is probably an internal helper function. No help will come up, because TMW does not intend for you to use it. I think this might just be a case where TMW does not want you looking under the hood. Another option is to look on the FEX for other users' implementations of the finite differences method and compare those to your own. They may not be as fast as the built-in function, but they will at least have all of their code available to you.
Good luck.

Sign in to comment.


Answer by Aaronne
on 20 May 2011

Hi Andy,
Thanks for your reply. Yes, I just try to dig out the 'fminunc.m' and find that it calls 'fminusub.m' to solve the medium scale optimization.
Then in the 'fminusub.m' it calls 'finitedifferences' to solve the gradient.
[gradFd,~,~,numEvals] = finitedifferences(x,funfcn{3},[],[],[],f,[],[], ...
1:numberOfVariables,finDiffOpts,sizes,gradFd,[],[],finDiffFlags,[],varargin{:});
However, the 'finitedifferences' is a '.p' file, and we can't see the real implementation. But any one knows how to use this 'finitedifferences' function? Because I use
help finitedifferences
give me nothing...
Thanks again for anyone who help me. Cheers, Aaron

  0 Comments

Sign in to comment.