John D'Errico

(I've fixed the bug about the outputfcn problem and re-posted it. The new version should appear sometime this morning.)

As far as Stefan's problem goes, anytime you throw problems with variables that vary by 20 to 30 powers of 10 at ANY numerical optimizer, expect serious things to go wrong. This is a no-no for literally ANY numerical tool. And setting a convergence tolerance (TolX) to eps is just as silly, especially when your parameters vary by that many orders of magnitude.

Essentially, you are asking that one variable, which can apparently vary somewhere between 1 and 1e30, must be computed to an absolute accuracy of roughly 2e-16.

A common solution is to normalize your variables, so that both are at least of similar orders of magnitude. If you were trying to solve a problem where one variable had bounds of 1e20 to 5e20, and a second variable was bounded in the interval 1e-6 to 1e-5, then normalizing the variables to both be of order 1 would make complete sense. But your variables each vary by many powers of 10!

One thing all novices need to learn about computing, is that when things vary by that many powers of 10, is to use logs! Let fminsearch vary the log10 of your variables, so they now have bound vectors that look like [0, -6] for the lower bounds, and [30, 0] for the upper bound. Now, inside your function, take the parameters and raise 10 to that power BEFORE they are used. Do the same with the output when it is returned. As far as fminsearch is concerned, your numbers are perfectly normal looking, but your objective sees the numbers in their proper scales.

You will find that any optimizer runs much more happily when you do this, as now the variables are very nicely normalized. All of the mathematics works more simply when you work in the log space. In fact, this is surely the natural way to work with variables that vary by many powers of 10. A nice consequence is the tolerances in TolX become relative tolerances on the variables, something that surely makes much more sense.

And DON'T use eps as a convergence tolerance. Fminsearch will never be able to give you even a reasonable shot at convergence in even 10000 function evals with that fine of a tolerance. So what happens is fminsearch will keep on iterating until it runs out of function evaluations. Use a meaningful, realistic tolerance. You were probably only setting that fine of a tolerance because of the ridiculous variable scaling anyway.

Willem - Personally, I think that 1.57982346157884756363579013421232435 is a slightly better value, although I am willing to negotiate on the last few digits. My point is, you are apparently trying to second guess the value predicted by less than 10% on a very uncertain value. How many angels can fit on the head of a pin anyway?

So feel perfectly free to pick whatever value you like there, when you write your own code. The value that was generated is the one consistent with the equations I specified, including the boundary conditions I chose. While those equations make some rational sense, I've won't claim that the predictions inpaint_nans makes are perfect, or that it will make all users blissfully happy. That would be impossible anyway, and I'm just too busy counting angels.

While I'm happy to see that some find VPI useful, nothing forces you to use a tool that is designed to operate on numbers with hundreds or even many many thousands of decimal digits. If your goal really is nothing larger than that will fit into a 64 bit integer, then use uint64 for your operations. (Only 64 bits is amazingly small in context of the problems VPI is designed to handle.) VPI obviously cannot compete in speed with arithmetic on those numbers, as this is an issue of apples versus oranges.

One day I expect to get around to re-writing VPI, but even then I will not reasonably expect more than about 10-1 speedup in any tests I've done, and even that will force some potential compromises on my part.

Use the right tool for your problem. In this case, uint64 seems like a better choice.

(I've fixed the bug about the outputfcn problem and re-posted it. The new version should appear sometime this morning.)

As far as Stefan's problem goes, anytime you throw problems with variables that vary by 20 to 30 powers of 10 at ANY numerical optimizer, expect serious things to go wrong. This is a no-no for literally ANY numerical tool. And setting a convergence tolerance (TolX) to eps is just as silly, especially when your parameters vary by that many orders of magnitude.

Essentially, you are asking that one variable, which can apparently vary somewhere between 1 and 1e30, must be computed to an absolute accuracy of roughly 2e-16.

A common solution is to normalize your variables, so that both are at least of similar orders of magnitude. If you were trying to solve a problem where one variable had bounds of 1e20 to 5e20, and a second variable was bounded in the interval 1e-6 to 1e-5, then normalizing the variables to both be of order 1 would make complete sense. But your variables each vary by many powers of 10!

One thing all novices need to learn about computing, is that when things vary by that many powers of 10, is to use logs! Let fminsearch vary the log10 of your variables, so they now have bound vectors that look like [0, -6] for the lower bounds, and [30, 0] for the upper bound. Now, inside your function, take the parameters and raise 10 to that power BEFORE they are used. Do the same with the output when it is returned. As far as fminsearch is concerned, your numbers are perfectly normal looking, but your objective sees the numbers in their proper scales.

You will find that any optimizer runs much more happily when you do this, as now the variables are very nicely normalized. All of the mathematics works more simply when you work in the log space. In fact, this is surely the natural way to work with variables that vary by many powers of 10. A nice consequence is the tolerances in TolX become relative tolerances on the variables, something that surely makes much more sense.

And DON'T use eps as a convergence tolerance. Fminsearch will never be able to give you even a reasonable shot at convergence in even 10000 function evals with that fine of a tolerance. So what happens is fminsearch will keep on iterating until it runs out of function evaluations. Use a meaningful, realistic tolerance. You were probably only setting that fine of a tolerance because of the ridiculous variable scaling anyway.

Great function, very handy.

I have noticed a small error, though. If fminsearch is not called, the function returns output.funcount, whereas it returns output.funcCount otherwise.