robilad at yahoo.com
Thu Mar 21 06:25:25 PST 2002
On Thursday 21 March 2002 10:24, Jukka Santala wrote:
> On Tue, 19 Mar 2002, Dalibor Topic wrote:
> > Speaking of benchmarking suites, there is Ashes from the SableVM people.
> > http://www.sable.mcgill.ca/ashes/
> > I think it's open source, so I'd prefer that. :)
> Okay, thanks, I'll take a look at that. There are also some open source
> conformance test-suites, I believe.
There is mauve. There is jacks (compiler conformance). Then there are the
regression test suites of various implementations, like libgcj's. I have run
kaffe with mauve. I haven't looked into other conformance test-suites.
> Also, it does give you a rough idea on conformance if the benchmarks break
> too badly :) I was thinking more in the terms of "breaking some
> optimization". The problem with optimizations is that they're quite
> platform specific and sometimes longer to run than simple conformance/
> regression tests, so they're difficult for a single developer to run. In
> addition Kaffe has pretty good conformance, but it would appear
> performance could use some work.
Optimizations in core & library code should be beneficial to all platforms.
But I guess you are not talking about those, since they are not platform
I don't think I understood the term "breaking some optimization" properly. Do
you mean "breaking some benchmark" ? If so, yes, meaningful benchmarks will
take longer to run than conformance tests in general. You have to run a
benchmark for some time to avoid measuring side effects like disk I/O, VM
startup etc. Unless of course that's exactly what you want to measure :)
Conformance tests usually run for a couple of seconds each.
In general, if people run kaffe on benchmarks & put the results on the web, I
assume that Jim will link to them. If people volunteer to do so regularly,
like the nightly builds done by the flex people, someone could write a couple
of scripts to collect & present the information in a nice way. All it takes
is to pick the benchmarks, find volunteers and agree on the procedure :)
I could submit results on x86-linux, interpreter/jit/jit3. But that's a rather
common platform, and not that interesting, I guess. Unless there is a massive
amount of interest in seeing how kaffe performs on old, slow computers :)
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
More information about the kaffe