Measuring code execution times is hard. Learn how to eliminate systematic and random measurement errors and obtain more reliable results.
We often need to measure how long a specific part of code takes to execute. Unfortunately, simply measuring the system time before and after a function call is not very robust and susceptible to systematic and random measurement errors. This is especially true for measuring very short intervals (< 100 milliseconds).
Systematic and Random Errors
So what is wrong with the following way of measuring?
time_start = time.perf_counter() my_function() time_end = time.perf_counter() execution_time = time_end - time_start
First, there is a systematic error: by invoking
time.perf_counter(), an unknown amount of time is added to the execution time of
my_function(). How much time? This depends on the OS, the particular implementation and other uncontrollable factors.
Second, there is a random error: the execution time of the call to
my_function() will vary to a certain degree.
We can combat the random error by just performing multiple measurements and taking the average of those. However, it is much more challenging to remove the systematic error.