[kaffe] 1.1.0 development release + ideas

[Jim Pick]

Hi,

Now that 1.0.7 has been released, we can get on with development.  I'd
like to leave the spotlight on 1.0.7 for a few days, and then I'll cut a
1.1.0 release from out of CVS (I won't test it a whole bunch, I'll just
make a release).

I'd like to see the 1.1.x releases happen every few weeks or so
(depending on the amount of development activity).

Eventually, we'll wind things up, and try to do a 1.2.0 production
release.  But I don't see that happening for a while (maybe 6 months). 
In the meantime, we could always backport a few changes and make more
1.0.x releases.

Some things I'd love to see happen:

* New build system

  - I've got a more modular build system semi-built that I designed
    for KaffePro.  It's very modular, bits are checked into different
    CVS modules, and combined using CVS's "ampersand modules" feature
    (in CVSROOT/modules).  There are separate modules for the core vm,
    io libs, core libs, security, networking, awt, command line, etc.

  - The first thing that happens in the build process is that a
    "bootstrap" version of kaffe gets built (for the host platform),
    using a fairly conventional configure/make type process.  After
    that, the build process is done from a (still somewhat crude)
    Java application, called "kaffe-buildtool".

  - kaffe-buildtool will support interactive configuration (similar
    to Linux's "make menuconfig" or "make xconfig"), so it will be
    possible to tune Kaffe to specific environments or applications
    (eg. cross-compiling subsets for embedded uses).  (WARNING!  I'm
    not a great Java programmer, please don't laugh at my code)

  - because the build process is Java driven, there is no need for
    Makefiles outside of the stuff needed to build the bootstrap.
    The logic currently encapsulated in libtool/automake/libltdl
    gets moved into the java build tool.  This will make maintenance
    much easier.

  - the class libraries will be built at runtime.  We shouldn't have to
    check anything other than a few bootstrap classes into CVS if we
    require the use of jikes (for releases, we can ship a pre-built
    Klasses.jar with enough classes to run kjc).

  - unfortunately, the build system I've got so far (that I did for
    KaffePro) is only partially complete, and isn't quite ready to
    replace what we've currently got.  I'd like to check what I've
    got in to get some feedback, but it's not good enough yet that
    it could be used to replace the current build system without
    breaking most ports.  So I'm going to do development on the build
    system in parallel with our development releases, until it's
    ready to be merged in.

* Automated regression testing

  - I had set up a multi-platform regression testing rig for KaffePro
    back at Transvirtual.  I'd like to do something similar (sort of
    like TinderBox) on kaffe.org.  The scripts will need to be somewhat
    different since the machines won't all be on the same network (my
    old scripts did most of the work over NFS).  Something like this
    would be invaluable, since it's there are so many ports.

* Classpath

  - With the more modular build system, and interactive configuration,
    it should be possible to incorporate more work from third-party
    free software projects.  Classpath is licensed as GPL+Exception, so
    we can use their code (on a case-by-case basis).  Future development
    of the class libraries could be directed to the Classpath project,
    so other VM's (eg. ORP, gcj, Kissme, etc.) would benefit as well as
    Kaffe.  For example, I'd like to see JavaSound implemented, so I
    could try JavaLayer (a 100% Java MP3 library) - it would be nice to
    do that via Classpath.  In some cases, Classpath won't be able to
    take some code (eg. pure-GPL code, like Transvirtual's AWT code) --
    we could host those libs ourselves on a mix-and-match basis.

* PocketLinux Kaffe merge

  - There was a lot of good stuff that Transvirtual released as part of
    their PocketLinux version of Kaffe.  For example, they had an AWT
    that ran on the framebuffer, as well as various changes to the core
    vm and JITs for MIPS and ARM that worked quite well, as well as
    ports to several proprietary RTOSes.  I'd like to migrate some of
    that work into our version.

* More ports

  - There are a lot of ports and patches that haven't been merged in
    yet.  I've got patches for SH and MIPS in my queue, and there are
    more exotic things like CRIS out there.  Plus there's a lot of
    bitrot that needs fixing, and some architectures that have working
    interpreters could use a JIT.  I'd like to see the Windows CE port
    go in, plus BeOS, and some of the RTOS's from the PocketLinux
    version.  It would be nice to see pthreads brought up to the same
    level as jthreads - maybe we could get some better interoperability
    with JNI code written for other VMs in that case.

* Documentation!

  - Kaffe is still a fairly simple VM to work with, and could be an
    ideal platform for people trying to do research or just trying to
    learn how a VM works.  Unfortunately, Kaffe doesn't come with much
    documentation.  There is some documentation out there, but it's
    scattered all over the internet.  It sure would be nice if we could
    check some nice documentation into CVS beside the source code.  I've
    got some ideas on how to lay it out, and I'm willing to write some
    of it.

* Precise garbage collection (non-conservative)

  - Kaffe's garbage collection works, but it's fairly crude, and it
    can't use relocating/copying schemes.  Work needs to be done on
    the code generated by the JIT and the native interfaces in order
    to enable us to build in more advanced, better performing,
    generational-style GC systems.  It would be nice if we could
    support some interoperability with plug-in GC modules from other
    free software VMs, such as ORP or Jikes RVM.

* gcj integration

  - Some work was done on allowing kaffe to use classes and methods
    pre-compiled to native code using gcj.  It would be nice to dust
    this off and update it.

* Specification writing / Mauve / Standards work

  - I've got some ideas for a project that would build a set of
    specifications for free software virtual machines, along with a
    compatibility test suite to certify virtual machines against
    those specs.  Unfortunately, Sun only certifies Java virtual
    machines through the Java Community Process (and charges $$$ for
    the JCK).  I don't see why we can't make our own set of specs
    that are meant to be interoperable with Sun's, but which are
    legally compatible with the needs of the free software community.
    We'd use Mauve as our equivalent of the JCK, and other VM
    implementations would be able to certify themselves against the
    spec.

* JIT improvements

  - Right now, our JIT is still pretty simple.  It would be nice if we
    could enhance it to allow for more elaborate compilation schemes
    (eg. optimizations).  In particular, I'd like to keep the
    compilation framework clean and modular enough that it could be
    used as a basis for research projects, etc.  Some other projects
    such as ORP and Jikes RVM have pluggable JIT modules - maybe we
    can be interoperable with those?  The whole concept of doing dynamic
    compilation at runtime is exciting, since it is theoretically
    possible to do a better job of machine code generation than what a
    static compiler can do - but that involves building some serious
    world class compiler infrastructure (a very big job).

* Verifier / Security

  - We don't have a full verifier, so we can't run untrusted code.

* Realtime Spec (JSR 1)

  - I haven't read the Realtime Specification for Java, but perhaps we
    could implement some of it.

* Process separation (JSR 121 - Application Isolation API Specification)

  - I read Godmar's thesis on KaffeOS (very nice), and found it very
    interesting.  Since JanosVM and KaffeOS are based on Kaffe, maybe
    JSR 121 isn't too hard to implement.  I haven't looked at it though.

* J2ME

  - With an interactive configuration system, we could configure Kaffe
    so that it would be possible to build it in various J2ME-compliant
    configurations.

* Mozilla integration / OJI

  - There were some patches that allowed Kaffe to be run inside
    Mozilla using the OJI interface.  That would be nice to see
    distributed with Kaffe.  Maybe we could do plug-ins for things like
    Konqueror and Internet Explorer too?

* Commandline / JNI invocation interface

  - I'd like to update the main program that handles the kaffe
    commandline so that it was smart about having multiple backends.  So
    kaffe could be built with multiple backend vms (eg. 
    libkaffevm-intrp-debug.so, libkaffevm-jit3-optimized.so), and
    the front end would dynamically link with the appropriate one at
    runtime based on the command line switches passed to it.  This could
    get complicated with native methods, since we might have to compile
    them multiple times, once for each vm.

* AWT improvements

  - It would be nice to merge in the framebuffer AWT from PocketLinux.
    There are some other AWT and AWT-like libraries out there that we
    could use as well (eg. Rudolph, Gtk AWT, QT AWT, Classpath AWT,
    SWT).  I'd like to see some aesthetic work done on the AWT.  :-)

* Debugging / Profiling

  - JPDA/JVMPI support would be very nice.  Also, gdb now does a nice
    job debugging Java code compiled with gcj - it would be nice if it
    could be made to work with an interpreter/JIT like Kaffe.  Plus
    there's the BDBJ debugger that works with Kaffe that looks very
    interesting.

* Various API updates

  - There are a lot of new Java APIs that we could support.  Some, like
    the new Classloader semantics, would be really nice.  If we get
    Classpath working, then some of this could be done via Classpath,
    but in some cases, the VM component needs updates as well.

I've probably forgotten some stuff ... I'm interested in what other
people would like to see.  I'll use this information to update the
"Projects" page on the website, when I get time.

I do plan to spend some time on some of these things this summer
(probably about a day a week) - I should be able to make some progress,
however slow it may be.

On a personal note, I haven't announced this to the list yet -- I no
longer work for Transvirtual, as I left to work for Berkeley Signal,
Inc. (BSI) about 3 weeks ago.  Transvirtual is still very supportive of
Kaffe.org (eg. they are providing the server) and approves of the
continuing work on the project.  I believe with my new job, I'll be able
to contribute even more to the project than I did in the past (I might
actually be using Kaffe to do some settop box prototyping). 

Cheers,

 - Jim