[kaffe] Future directions for Kaffe

[Dalibor Topic]

Jim Pick wrote:

>Hi everybody,
>
>  
>
Hi Jim!

>It's been a quiet month on the mailing list so far.  That's partly my
>fault, I think, since the mailing list was broken for some time.
>
>  
>
Thank you for fixing it again.

>It looks like the last CVS commit was 5 weeks ago.
>
>I see Dalibor went to FOSDEM, and did some talking about Kaffe there.
>  
>
Yes, there is a small write up on 
http://robilad.livejournal.com/8458.html .

>So I'm assuming that the project isn't dead, it's just somewhat dormant.
> It's been somewhat dead/dormant throughout much of it's history, but
>it's still here, isn't it?  :-)
>
>  
>
Indeed :)

>And clearly, all the free Java runtimes and Classpath are in a state of
>transition, as we wait for Sun to release the rest of OpenJDK.
>
>I'd like to liven up the list a bit, and maybe start a bit of discussion
>on where Kaffe should go next.
>
>Here are some things I'd like to talk about:
>
>* I definitely need to do some work on upgrading the server, and fixing
>up the website.  Currently it's running a really old version of Debian,
>so it needs to be upgraded.  I'm just scared of all the breakage that
>will happen.  I'm slowly building up my hosting capabilities, but it's
>just a hobby, and I have real life things going on, so I move at a
>glacial pace.  If anybody wants to help out with any of that, I'd really
>appreciate it.  I happy to keep hosting it indefinitely.
>
>* I think a wiki running on top of Kaffe would be really nice.  :-)
>
>* On the other hand, there are establishing free software hosting
>platforms like Sourceforge, Savannah, Google Code, etc. that might
>work better than just running everything on our own server.  Our current
>infrastructure is pretty much still using technology from the 1990s.  We
>don't even have a blog or a wiki, or any continuous integration or
>distributed version control.  I'm open to migrating things if that's
>what people would prefer.
>  
>
You've done a great job keeping the infrastructure together and alive. 
Thank you for all the good
work on keeping the playground in a good shape, Jim, and for providing 
the place for us to play in.

As far as migrating things goes, I think that mostly depends on how much 
additional infrastructure
work one could realistically expect to be done by volunteers. I'm rather 
sceptical about that, since
infrastrucural work is not really sexy for new developers. The 'the web 
site could be improved by
doing x,y,z' feedback I received quite often over the past years has 
very rarely been followed up
by actual patches, or someone volunteering to revamp the site.

In that light, I'd recommend 'outsourcing' additional services to 
external infrastructure providers, i.e
java.net, Google Code, etc. simply because it removes the future 
maintenance burden from the
equation.

>* Technically speaking, I'm still the project leader, by virtue of
>rescuing it from the ashes of Transvirtual.  But Dalibor is really the
>guy who has been doing most of the work.  I'm not really doing much with
>Kaffe personally, so if anybody else wants to step up and be a real
>project leader, feel free to volunteer.  I'm still happy to keep hosting
>the project and helping out with the releases.
>
>  
>
I don't use Kaffe (or Java, for that matter) for my personal work (SPASS 
is written in
C & C++, and my area of research has nothing at all to do with JVMs), so 
my own
priorities would be quite different from someone using Kaffe for their 
personal work,
I'm afraid.

In the past five years I've used Kaffe as a playground for ideas for 
acceleration of the
liberation of the Java platform, as well as to learn about new tools, 
and make new connections.

I think it has worked out great as a soapbox, allowing me to formulate 
ideas and bring them
into the mainstream of the regular Java audience, thanks to Kaffe being 
a strong 'brand', while
at the same time pushing the envelope of what we can tie together under 
the umbrella of a
'free software SDK  for the Java programming language', and allowing 
distributors to use it
as a runtime until gcj took over that role.

So, as a 'political platform', Kaffe has been incredibly successful, I 
think, as what started out as
waking up a dormant project back in early 2002, ended up being one of 
the projects that crucially
pushed the boundaries on what was thought to be possible for free 
software JVMs, and
bulldozing a path forward until things started to get a lot better in 
the Java world.

Part of the strategy going from late 2002 on was to avoid the problems 
of the 90s, where Kaffe's
parent company and gcj's parent company were engaged in a fruitless 
effort to compete with
each other in the embedded space, leading to a lot of friction with 
little to show for it. So there
was no point in trying to re-launch Kaffe as a contenter for the 'crown' 
of the free JVMs, as
that would have just led to friction over the scarce volunteer developer 
resources, rather than
'lifting all boats'. GNU Classpath turned out to be the project that was 
lifting all boats, and
the winning strategy turned out to be to make it as easy as possible for 
people to start their own
VM projects while being able to reuse as much as they can from a common 
'incubator' of
technology, and to demonstrate the demand for 'Java libre', dissuade the 
fears about evilness of
open source JVM implementations,  and show how to get there without 
running into (legal)
trouble.

As should be clear from the above, my goals over the past few years 
involved a lot of soft work,
communications, diplomacy, etc. rather than making Kaffe excell as a 
'technical platform'. In my
opinion, technical excellence was a goal that made no sense to run after 
in a project without a
strong academic or commercial backing, and in both cases, the project 
had successful runs that
were unlikely to be repeated in the future, since it had technologically 
fallen behind Sun's
implementation by a long shot by the end of 2002, basically being stuck 
in 1998 both wrt to the
class library and to the core VM. So the strategy there was to try 
solving the class library issue first,
by migrating away from Kaffe's old class library slowly but surely, as 
GNU Classpath caught up
with it in the areas where Kaffe was still ahead. Trying to fix both the 
core VM and the class libraries
at the same time would have been too much work for a project relying 
solely on individuals in
their spare time to help it move ahead. It makes more sense to me to 
have a slow VM that can
run Eclipse, rather than a fast VM that can run HelloWorld.

With the switch to GNU Classpath being 99,9% completed since last 
autumn, that part is now done,
and the road is open to fixing the core VM. So, where should the core VM 
go to in the future?

I'm not an enthusiastic low level hacker myself, I prefer C to assembly, 
C++ to C, and Java to all
of them. In that regard, Kaffe has had remarkable success anyway 
attracting people with the necessary
skills to do the necessary work on the core VM, like Guilhem is doing 
now. Unfortunately, all of us
on the core team have other, real lives beside Kaffe, so I don't think 
that it's realistic that Kaffe can
improve the core VM in a way that makes it competitive again with 
HotSpot, based on what we have
now. It would take too much work for a gang of volunteers to reimplement 
all the things Cacao has
done, for example, without having an academic institution behind Kaffe 
to supply all the 'volunteers'. :)

So, in my opinion, the way forward there is to use code from other (J)VM 
projects as much as possible,
rather than (re)writing our own. I'd be interested in wielding a chansaw 
after the next release, in order to
get rid of as much of the code in Kaffe as possible, and to replace it 
with code from other sources that we
can link to, rather than having to maintain it.

>* Speaking of releases, we really should do another release sometime.
>  
>
Yeah, my bad. I didn't find the time & motivation to tie all the lose 
ends yet. Otoh, I think we should just mark
the failing tests with XFAILs, and ship it, so that we can have a cesure 
point for people struggling to
build the whole thing with the latest gcc.

I've released a separate version of fastjar meanwhile, though, so that 
we can kick it out of Kaffe again, though. :)

>* I'm embarrassed to admit that I don't even have Kaffe running on my
>new MacBook under OS X.  I got it to compile, but I couldn't get it to
>even run "Hello World".  If I spent some time on it, I imagine I could
>figure it out.  I just haven't spent the time.  I hope it still runs OK
>on Linux, but I haven't tried that recently either.
>  
>
You'll need to use libffi on x86 OS X.

>* I also haven't been responding to emails asking me for help getting
>Kaffe to run.  I'd like to, but since I don't even have it working for
>myself, I'm not really in a position to help out.  I get so much spam
>nowadays that I hardly even use email anymore.  I notice that most
>requests for help to the mailing list are going unanswered as well.
>
>  
>
Yeah. I think that's largely since they are somewhat misdirected, as for 
many of the things people seem
to want, there are better alternatives out there by now:

* If you want to do research on a C-based JVM, you should get in touch 
with the Cacao team.
* If you just want to do reseach on a JVM, you should get in touch with 
the JikesRVM team.
* If you want a JVM for your embedded linux system, you should get in 
touch with the PhoneME team.
* If you want a fast  interpreter, you should get in touch with the 
JamVM developer.
* If you want a blazing fast, fully featured free JVM, you should get in 
touch with the HotSpot team.

Kaffe's current niche is

* You want to support a non-mainstream platform
* You enjoy playing around with low level C code
* You want to learn about C & autotools development
* You want to move the earth, and need a place to stand :)

>* I'm still interested in playing with Java virtualization, and I'm very
>excited about OpenJDK coming out.  JRuby looks really interesting to me.
> For my own projects, I'm guessing I'd probably use OpenJDK in
>preference to Kaffe in the future, since it's likely to be a lot less
>effort to get it working the way I want it to.
>
>* That said, I think Kaffe has been a seminal project in terms of
>getting free Java off the ground, and I'd hate to see it die.  A lot of
>interesting projects have used Kaffe as a starting point.
>
>* I imagine that in the future, people will most likely look to OpenJDK
>as a starting point to add their enhancements.  Is there still a role
>for Kaffe to play here?
>
>  
>
I don't think there is a realistic role there, since OpenJDK has a 
research oriented community as well,
so I'd only expect that to bloom further with time, as it is pretty 
attractive in terms of papers one
can produce to apply research to a production quality VM.

For other purposes, both JikesRVM and Cacao offer existing communities 
doing research in the field,
while Kaffe, as a project, does not. I don't think there would be much 
of a point, anyway: the research
infrastructure JikesRVM or Cacao can provide beats what we could offer 
without ties and access to
academia.

>* I think Kaffe probably is still the simplest full JVM implementation
>that isn't just an interpreter.  It's been used for all manner of exotic
>porting projects that might just be too hard to do using something like
>OpenJDK or gcj.
>  
>
Yeah. On the other hand, we have been rather bad at integrating that 
work back into the main tree.
As time passes on, I think we should write off the chance of ever 
merging in the work that was
done in many of the research forks, since the people behind those 
projects have moved on to do
other things, and Kaffe's internals have changed quite a bit since 2002.

>* Kaffe is licensed under the GPLv2.  So is OpenJDK.  But Kaffe doesn't
>require copyright assignment, and we're pretty open.  Sun doesn't have
>to vette the code going into Kaffe.  That suggests that perhaps we could
>merge in large parts of OpenJDK, and provide a place for people to do
>really experimental stuff that Sun isn't going to permit in their
>version.  Is this something we should consider?
>  
>
I don't think that's a viable, long-term direction. In our discussion 
with OpenJDK engineers at
FOSDEM, I tried to make sure that they understand what our experiences 
were with the kaffe model
(let them fork, and wait to see if they ever come back), and the gcc 
model (let them work on branches
in the same repository). I think wrt to integrating innovative changes 
back into the main tree, the
gcc approach has shown to be more successful, and Sun's approach to 
encourage innovation
on the Java programming language using the Kitchen Sink Language project 
to branch javac
on java.net suggests that they understand that.

In that light, and since OpenJDK will use a distributed source code 
versioning control system,
I doubt that that will be a viable niche.

On a more technical ground, HotSpot does not necessarily lend itself to 
being easily ripped apart
and merged gradually into Kaffe, since it's written in C++, without the 
goal for the components to be
easily reusable. We should be able to merge in the verifier, but I would 
not expect to be able to
merge in other components without doing some work on HotSpot first, and 
it's not clear to me what the
benefit in that case for HotSpot would be.

>* In other words, should we go big?  And merge in as much stuff as
>possible.  That could be problematic, since Kaffe is already pretty
>huge.  Maybe we could adopt more of a "distribution" approach, and break
>things into a bunch of modules that are all developed to work together?
>
>* Or maybe we should try to stay small?  And just try to be an easily
>hackable, simple virtual machine with a crude compiler framework, and
>nothing else?  That would involve jettisoning or spinning out a lot of
>the integration work that's been done over the last few years, I think.
>
>* I think we've been trending towards the "go big" direction for some
>time, with all the Classpath merging and other projects, and the core
>has been somewhat neglected.  It's been really good to support Classpath
>this way, and it's helped to get a lot of Java stuff integrated into
>Debian.  On the other hand, I think the build itself is just too
>intimidating.  It's massive.  I think most prospective developers would
>probably give up before getting it to build the first time.  Our
>configure scripts are almost an operating system in and of themselves.  :-)
>
>  
>
Indeed. Going big was necessary as long as the 'bulldozing the road to 
free Java' was necessary, and
most of the code we used was not packaged by *BSDs, Linux and other 
operating system distributions.

Both of these things have changed over the past year, with even GNU 
Classpath being packaged everywhere,
thanks to JamVM's, IKVM's & Cacao's ubiquity, so that we can start 
ripping out code. I think the future
for Kaffe is in wielding a chainsaw to first remove as much code as 
possible, trimming down the
forest of configuration options, and then to replace as much of the core 
VM code by code from other
actively maintained projects. So I'd like to go small in the future, 
since I don't see where Kaffe would find
the ressources for going big. :)

Technically speaking, Kaffe's core VM is at least 5 years behind Sun in 
terms of features and performance,
and I don't see us being able to close that gap in the future without 
delegating some of the core functionality
to other projects, like LLVM, and letting glib, gnet etc. shield us from 
all the portability cruft that's been
accumulating in the configure script.

In my case, I'm interested in a few things in the coming year: seeing 
OpenJDK succeed, seeing the ecosystem
around GNU Classpath collaborate with Sun on Java 7, seeing the 
deployment problems of Java resolved, and
seeing the JCP remove barriers to participation. Kaffe can help me get 
some of these things done, so I'll keep
hacking on it in order to make it all work out. But the work outline 
above in terms of jettisonning the existing core
VM code, and replacing it with something more maintainable today, would 
be a good opportunity to
attract a new generation of developers wanting to make their own name. :)

cheers,
dalibor topic