TIMEIT Benchmarking Function

TIMEIT.M measures the time required to call a user-specified function

Description

T = TIMEIT(F) measures the time (in seconds) required to run F, which is a function handle.

TIMEIT handles automatically the usual benchmarking procedures of "warming up" F, figuring out how many times to repeat F in a timing loop, etc. TIMEIT uses a median to form a reasonably robust time estimate.

UPDATED 31-Dec-2008: More accurate when timing very fast functions; warns you when the reported time might be affected by time-measurement overhead; calls F fewer times when F takes more than a few seconds to run.

• MATLAB release: MATLAB 7.5 (R2007b)

[John D'Errico]

Ok, this does nothing exotic. But what it does it does well. Its a simple way to time your code without worrying about some of the extra bookkeeping, like warming up the codes to make sure you get an accurate assessment of the time required, plus taking the median of the time over multiple runs.

The help and the code is as good as you would expect from this source.

[Anna Nowak]

Small remark - nargout parameter is sometimes less than 0. I had -1 result when there was no output arguments from function.

In case of problems with error "Too many output argument" juz change line :
num_outputs = ~(nargout(f) == 0);
to
num_outputs = ~(nargout(f) <= 0);

Still excellent file!

[Steve Eddins]

Anna - thanks. I just updated the file to fix the nargout problem.

[Paolo de Leva]

A suggestion to improve accuracy when you measure small execution times. I believe it would be useful to subtract from median(times) the time needed for the inner iterations (“offset”). Here is the code:

[…omissis…]
offsets = zeros(num_outer_iterations, 1);
times = offsets;
for k = 1:num_outer_iterations
    t1 = tic;
    for p = 1:num_inner_iterations
    end
    offsets(k) = toc(t1);
end

for k = 1:num_outer_iterations
    t1 = tic;
    for p = 1:num_inner_iterations
        [outputs{:}] = f();
    end
    times(k) = toc(t1);
end
 
t = (median(times-offsets)) / num_inner_iterations;
[…omissis…]

This offset is not always negligible, especially when you want to time the execution of very quick commands or functions. Notice that the value of each element of variable “offsets” also include the time needed to execute

t1 = tic; times(k) = toc(t1);

On my system, this is very small (1.7321e-005 s), but not negligible when testing the execution of very quick commands, such as RESHAPE.

Paolo de Leva

[Steve Eddins]

Paolo, thanks for the suggestion. I'll look into it.

[Joao Henriques]

This is extremely useful, but since it requires a function definition, you must either use lambdas, which don't allow procedural code (ie, a list of statements), define a local function, which means you have to turn your script into a function file (sometimes not desirable when dealing with workspace variables), or create a new file (lots of throwaway files with simple routines).

I wonder if it would be possible to add the following functionality (either in the same function or through a different one) :

while timeit %returns true when a new measurement is needed
  ... my procedural code ...
  ... multiple lines, will run as many times as timeit() sees fit ...
end
result = timeit; %next call: return result. reset state.

As you can see, this sort of usage would be tremendously useful to just profile a piece of code with no extra details -- just wrap it in a loop. Of course, it's only fair if you conclude it's not worth the additional effort :)