Computational time evaluation over different computers

Hi,

I have the following problem. I built 2 functions: one should be faster, and let's call it F1, and another one should be slower, let's call it F2. I have 2 computers, one is a laptop with 8GB of RAM and one is a desktop with 12GB of RAM, and F1 is actually faster than F2. My colleague has also a laptop with 16GB of RAM (the processor is comparable to mine), and a desktop computer with 48GB of RAM. With his computers F1 is slower than F2.

The time is computed using timeit.

Now, I can accept that, between different computers, the time changes, but still the ratio should be the same: if something is programmed in a way that it should be faster, it should be faster with every machine. F1 should always be faster than F2.

With 4 machine more or less comparable in terms of processors, with different RAM, 2 machines give me the answer "F1" is faster, and 2 machines give me the answer "F2" is faster. Actually, the ones with more RAM give the result that F1 is the slowest one. All of them have the same MATLAB version

What can be the reason for that?

[Adam]

It's difficult to comment really without knowing what the two functions are and what size of data they are running on etc, etc.

For example, maybe the extra RAM allows F2 to be more efficient than it could be with less RAM whereas for F1 it makes no difference. Quite unlikely, but without knowing anything about the functions it is impossible to say.

Answer 1

[John D'Errico]

This is impossible to know, at least based on what you have given us for information.

It is not only tha RAM present. It may be the number of processors, the number of cores, bus speeds, amount of cache memory present. If it involves graphics, then the graphics card will obviously be critical, as well as the amount of RAM for that card. Some machines have solid state drives, and if there is any amount of virtual memory swapping done, that will be critical.

Is one machine running MATLAB installed on that machine, whereas another is running with files on a distant networked drive? This can be a huge speed penalty.

Are there processes running in the background? Is one of those machines infected with a virus? How can we know what the differences are that you have not disclosed?

So I'm sorry, but it is simply NOT true that some arbitrary script will ALWAYS be predictably faster on some arbitrary machine.

[Sean de Wolski]

But what is ALWAYS true is that it's depressing when your brand new computer runs a script more slowly than your old computer :)

[Maria]

But then, when in a paper for example, the execution time is presented, it has no sense. The only scientific way to compare function is the O notation, and that makes sense. But, again, there is then no sense when, in a scientific paper, they provide the execution time as proof for the great performance of a function in respect to another one, since they are playing with dices!

[Adam]

You always have to take run times with a pinch of salt in papers. They are almost always necessary, but can be misleading. When I did my PhD my programming skills (C++ then, but same for any language) were up to the task of getting the maths right and putting it into some kind of structures and functions, but that was about it.

I dread to think how badly I programmed the algorithm whose execution time I had to add to papers. It likely made the algorithm look bad in theory when mostly it was my inept programming. So yeah, there will always be factors in such timings that make them unreliable and differences from one computer to another probably not amongst the greatest of those!

With close to 10 years (on and off) more programming experience under my belt now than back then I could probably program the same algorithm many orders of magnitude faster.

All you can do is run the algorithm, give a time and give relevant aspects of the computer specs that gave you that time. Whether or not that is useful to someone reading it is another matter!

[Maria]

I see, you are right. Still I would expect that a function better then another in theory, it is still better than another, on every pc. Actually now, luckily for me, the "best" function is again the "best" function between me and my colleague. There was a wrong allocation in the memory inside a for loop, but on my pc this bad allocation did not affect the comparison between the two functions, but in my colleague one obviously it affected so much to make the "better" function performing worst.

Answer 2

[Kevin Dupraz]

Hi all,

I have a very strange behaviour with a simulation code under Matlab that is similar to this post.

My simulation code run on my laptop (Intel Core i7-4810MQ @2.8GHz, RAM 32 Go, windows 8.1 pro 64bits) and take 15s to finish in average (with matlab 2015b and 2017b).

I send the same code (classes and script) on the calculation server of my lab (Intel Xeon E5-2680 v2 @2.8GHz, RAM 128 Go, windows Server 2012 R2 Standard). On the server the code take twice the time to execute, hence about 30s (with matlab 2015b and 2018b).

On the paper the server is better than my laptop, so why the same code take more time (a factor of 2) ? There exists configuration to speed up the excution that is activate on my laptop and not on my server ?

Thanks for any explaination ^^.

PS: the timing is made with the "Run and time" button.