Odi's astoundingly incomplete notes

I have some JUnit test that exercise and compare the performance of certain classes. They usually look like this:

long start = System.currentTimeMillis();
test1();
long end = System.currentTimeMillis();
long duration1 = end - start;
start = System.currentTimeMillis();
test2();
end = System.currentTimeMillis();
long duration2 = end - start;
assertTrue(duration1 > duration2);

This is useful to detect performance regressions. During development of these tests I came across a very obvious but interesting fact: do not use differences of the system timers for these tests. With system timers I mean System.currentTimeMillis() and System.nanoTime().

Rather the ThreadMXBean.getCurrentThreadCpuTime() method should be used. This makes the test independent of the load on the system, most of scheduling artifacts and even the garbage collection. All in all I get very stable and consistent result from this accounting timer. So my tests now look like this:

ThreadMXBean mx = ManagementFactory.getThreadMXBean();
long start = mx.getCurrentThreadCpuTime();
test1();
long end = mx.getCurrentThreadCpuTime();
long duration1 = end - start;
start = mx.getCurrentThreadCpuTime();
test2();
end = mx.getCurrentThreadCpuTime();
long duration2 = end - start;
assertTrue(duration1 > duration2);

posted on 2008-06-02 15:47 UTC in Code | 0 comments | permalink