Performance
This will be a status page for tracking how Kaffe compares, performance-wise, to other Java virtual machines.
Remember, many benchmarks only test a specific task, severely stressing a particular area, such as number crunching or string manipulation. But remember, most real-world Java applications occupy the highest levels of the programming abstraction foodchain. Most are primarily concerned with user interfaces (eg. AWT or HTML generation), leaving the serious number crunching to native libraries and applications running in subprocesses. Applications are often bound by user-interface (waiting for user input) or I/O constraints (waiting for network packets, disk I/O, database queries, etc.), so the dramatic variations between different virtual machines you see in the benchmarks are often not reflected in real world performance. As they say, beware of "bench-marketing".
If you know of any benchmarking work or articles, send them to jim@kaffe.org.
Eventually, we'll build our own performance testsuite.
Benchmarks
- Ashes
A set of free benchmarks from the Sable project at McGill.
- jMocha
An open source microbenchmark framework and suite from IBM.
- SciMark
A Java benchmark for scientific and numerical computing.
- SPEC JVM98
A proprietary benchmark suite. Costs $100 ($50 for academic use) per person.
- RMI Benchmark Suite
Benchmark Competitions
- Shudo.net - Performance comparison of JITs (Jan 2002)
Blackdown JDK 1.3.1 (Hotspot Client VM, Server VM, sunwjit.so, OpenJIT, TYA, shuJIT, interpreter), IBM JDK 1.3, ORP 1.0.9, Jikes RVM, Kaffe, gcj
- IBM developerWorks: Weighing in on Java native compilation (Jan 2002)
gcj, Kaffe, Sun JDK, IBM JRE
- VolanoMark (March 2000)
Kaffe was benchmarked up until March 24, 2000.
- JavaWorld: The need for speed, stability (VolanoMark 2.1.2) (October 1999)
Tower TowerJ 3.1.4 Linux, IBM JDK 1.1.8 Windows NT, IBM JDK 1.1.8 OS/2, Microsoft VM 3229 Windows 2000, IBM JDK 1.1.8 Linux, Sun HotSpot 1.0.1 Windows NT, Sun JDK 1.3 Beta Windows NT, Sun JDK 1.2.2 Windows NT, Microsoft VM 3186 Windows NT, Sun JDK 1.2.1_04 Solaris, Novell JDK 1.1.7 NetWare, Sun JDK 1.2.1_03 Solaris, Sun JDK 1.2 Linux, Transvirtual Kaffe 1.0b4 Linux, Blackdown JDK 1.1.7 Linux, JDK 1.1.8 FreeBSD
- gcj performance: benchmarks (Aug 1999)
IBM JDK 1.1.6, gcj, JDK 1.2, Kaffe, JDK 1.1.7, gij
- JavaWorld: The Volano Report: Which Java platform is fastest, most scalable? A JavaWorld exclusive! (March 1999)
IBM JDK 1.1.7 OS/2, Tower TowerJ 2.2.7 Linux, IBM JDK 1.1.7 Windows, Microsoft SDK 3.1 Windows, Sun JDK 1.2 Windows, Novell JDK 1.1.5 NetWare, Sun JDK 1.2 Solaris, Blackdown JDK 1.1.7 Linux, JDK 1.1.7 FreeBSD, Kaffe 1.0 Linux
- JavaPro: VM Roadtest: Getting More Than You Pay For (Dec 1998)
Kaffe, Toba, JDK 1.1.6
- Harissa benchmarks (1998?)
Harissa, Toba, JDK 1.2b4, Kaffe
Here are some benchmarking links that don't involve Kaffe:
Here are some links to general literature on the problem:
|