Java SE 8 was released last month (March 18, 2014). Traditionally, every new major JRE version comes with a free performance boost. Do we get another free lunch? And how big is the gain this time?
Let’s benchmark it.
- Run the same code with 3 different JRE versions (SunJDK
1.8.0). The code itself was written for Java 6 (both in syntax and JDK API’s usage) and compiled for Java 6 with OpenJDK 1.7.
- Each run takes about 55 minutes.
- VM arguments:
Intel® Xeon® CPU W3550 @ 3.07GHz
- Each run solves 13 planning problems with OptaPlanner. Each planning problem runs for 5 to minutes. Up to 2 planning problems are solved in parallel.
- Solving a planning problem involves no IO (except a few milliseconds during startup to load the input). A single CPU is completely saturated. It constantly creates many short lived objects, and the GC collects them afterwards.
- The benchmarks measure the number of scores that can be calculated per millisecond. Higher is better. Calculating a score for a proposed planning solution is non-trivial: it involves many calculations, including checking for conflicts between every entity and every other entity.
To reproduce this benchmark locally, build optaplanner from source and run the main class GeneralOptaPlannerBenchmarkApp.
- On the biggest dataset (Machine Reassignment B10), which dwarfs any of the other datasets in size, Java 8 is
20%faster than Java 7, which was already
17%faster than Java 6.
- In some cases, Java 8 is slower than Java 7. Specially for the course scheduling datasets, Java 8 is
6%slower than Java 7. Hopefully new releases of Java 8 will resolve this performance regression soon.
- On average, Java 8 is only
1%faster than Java 7. This while Java 7 is already
16%faster than Java 6.
- Despite that this is the first final release of OpenJDK 8, I did not find any regressions in Java 8. OptaPlanner’s examples are 100% reproducible, so as expected, the different JRE’s give the exact same results at every single iteration.
Raw Benchmark Numbers
On the big datasets, Java 8 is clearly faster. And this without changing a line of code. On average, the result is less convincing (with the current release), but a free lunch is always welcome.
This article is originally published at OptaPlanner Blog, re-posted here with authors permission and as part of the JBC program.