Testing concurrent code question 166 of 180 senior
How would you benchmark two concurrent implementations honestly?
- Time a loop with
System.currentTimeMillis - Correct: Use JMH: warm up the JIT, several forks, report the error bars, and measure under the contention level you actually care about
- Run it in production and watch the dashboard
- Count the lines of bytecode
A hand-rolled timing loop measures the interpreter, then the C1 compiler, then dead code the JIT removed because the result was unused. JMH exists to remove those: warmup iterations, blackholes for results, several forks to shuffle JIT decisions, and a reported error interval.
For concurrency add the parts specific to it: @Threads or a thread group per role to set the
contention level, @State(Scope.Benchmark) for the shared object, and separate numbers for one thread
and for N, because the interesting result is the shape of the curve rather than a single figure.
Say the honest part too: benchmark results are workload-specific, and two claims in the sibling repository's README turned out to be wrong once measured, which is why they now carry numbers.
Quiz yourself on Testing concurrent code Suggest a correction