[kaffe] Improving regression testing
robilad at yahoo.com
Fri Feb 28 09:42:01 PST 2003
--- Timothy Stack <stack at cs.utah.edu> wrote:
> > b) When you're cross-compiling, the version of
> > you're building
> > most probably can't run on the build system. That
> > means 'make check'
> > is not very useful.
> For the JanosVM we added a 'remote test' mode to
> TestScript. Basically,
> you would run 'make check' on a FreeBSD/Linux
> machine, but the tests were
> actually run on a remote machine running the JanosVM
> on bare hardware.
> Each test loop worked like this:
> compile class file locally
> upload file to test host
> execute test on test host
> download result files
> compare actual results against the expected ones
> This reduced the requirements on the test host since
> it doesn't require
> make, diff, javac/jikes, etc...
That sounds very good. a) was not really a big issue,
and that solves b) elegantly. I'll try to get it
> > If kaffe optionally installed the test script and
> > test jar. you
> > could run the tests on your target. That *should*
> > cross-compiling
> > developers to find & fix kaffe's bugs easier.
> Well, you still need the source files to compare the
> output, it just seems
> like a lot of baggage to install.
Yeah. I didn't think about the client-server approach,
you are using. It's the better solution in most cases.
> I added the 'internal' test directory to the JanosVM
> for exactly this
> purpose. Although, the only test in it at the
> moment is for the jitter.
> I would also like a 'profile' test directory that
> did some basic
> measurements of kaffe's behavior. For example, how
> much memory does it
> use to run HelloWorld or measured how many times a
> method could recurse
> before a StackOverflowError was thrown.
sounds good to me. I'll get the jitter tests merged
in. there were some include file name changes the last
time around I tried, which broke the C part of the
tests. I'll follow them in kaffe's CVS to make further
core merges with Janos VM easier.
> I'd hesitate to make too many changes, the current
> stuff works well
> enough, it just needs to be fleshed out a bit.
O.K., I'll give it a try. Being able to use prebuilt
tests might matter though, when the host compiler
miscompiles them (see the JanosVM Preempt test, which
breaks because of a kjc bug ).
As you've apparently done most of the hard work
already in JanosVM's regression test code, I'll use
that instead of reinventing the wheel. Thanks!
 on a side note, I've received a mail from a kjc
developer that he has confirmed the "kjc vs. mauve
Thread test" compilation bug, and is working on it.
Do you Yahoo!?
Yahoo! Tax Center - forms, calculators, tips, more
More information about the kaffe