Different Results in Identical Deterministic Calculations

Hi,

I am running a simulation program, which at the moment is not parallelised nor is it running on a GPU or anything. I use Matlab 2013a 64 bit (but observed the same for 2009a) on a Suse Linux system with 8 processors. The program is a multisclice scheme, that basically does multiplications and ffts. It is however, fully deterministic, hence same input should yield identical output. Nevertheless, I observe slight differences in the output. In matrices, whose entries are of about 1e-2, I get average deviations of ~1e-15 in consective executions. I relaise, that this is very small and irrelevant for the actual result, but what I was trying to do was to write a script, that will check the results of the code after changes to identitfy possible bugs. But now I am running into the problem, that two results of the same code are not even identical most of the time, even though sometimes two results are fully identical. I am aware, that for practical relisation this can be solved by some threshhold, but where does this deviation stem from?

I see, how differenet architectures or even processor types could cause this, but I am running the same Matlab on the same computer with 8 identical processors. Is this somehow expected?

With best regards, Hans

Answer 1

[John D'Errico]

This question comes up repeatedly. Invariably, the code is not in fact deterministic, or the person is doing something that causes those errors, without realizing what they did. But we have not been given any information about what it is that you are really doing. The mind-reading toolbox is still weeks away from release, so I am forced to rely on my trusty (but often foggy) crystal ball.

For example, I might guess that in fact, you might be using a tool like svds or eigs, and expecting them to yield fixed deterministic results, down to the last digit. Not so, since they use a random start point.

Or, you might not realize certain other facts about floating point numbers. For example, you might assume that certain basic rules of arithmetic, like the distributive law, that they truly apply to floating point arithmetic. It does not exactly apply.

Or, you might be copying numbers across systems, and hoping that the numbers were copied exactly, but your copy was not one where all of the binary bits were in fact transmitted.

No, I'm not thinking that any of the above examples I gave above are what you are doing. Odds are, from the very limited amount you have allowed to slip out, you are making some other error of assumption, but there is simply no way to know. You have not actually given us any useful information here on which to proceed.

So, yes, you can indeed have subtle deviations in a result, under certain circumstances. I've said very often that floating point arithmetic is not in fact actual mathematics. It looks like it is. The two domains use many of the same operators, many of the same "rules". But floating point arithmetic, especially when done on a modern CPU with multiple cores and using the BLAS, as much as it looks like mathematics, is only an approximation to the real thing.

Sorry for making such a vague response. But your question is undeniably vague, and my blasted crystal ball is just so foggy today. Perhaps if I try some crystal ball polish and window cleaner, it will help ... Nope.

[Walter Roberson]

As a practical point: experiment with setting the random number seed to a fixed value at the beginning. If you are somehow using random numbers without realizing it (e.g., the svd or eigs examples John mentioned) then setting the seed would at least force it to use the same random numbers, which would lead to identical results.