The obvious answer is Jikes RVM (Research Virtual machine).

"Jikes RVM (Research Virtual Machine) provides a flexible open testbed
to prototype virtual machine technologies and experiment with a large
variety of design alternatives. The system is licensed under an OSI
approved license. Jikes RVM runs on many platforms and advances the
state-of-the-art of virtual machine technologies for dynamic
compilation, adaptive optimization, garbage collection, thread
scheduling, and synchronization. A distinguishing characteristic of
Jikes RVM is that it is implemented in the Java™ programming language
and is self-hosted i.e., its Java code runs on itself without requiring
a second virtual machine. Most other virtual machines for the Java
platform are written in native code (typically, C or C++). A Java
implementation provides ease of portability, and a seamless integration
of virtual machine and application resources such as objects, threads,
and operating-system interfaces.

"Many researchers have found that Jikes RVM provides a useful vehicle
for research on the frontiers of virtual machine technologies (over 150
publications and 27 dissertations), as well as teaching courses. If you
are looking for where to start reading about the Jikes RVM we have a
recommended reading page."

http://jikesrvm.org/

Tom Hawtin

see http://mindprod.com/jgloss/jvm.html

Sun releases the source to its stuff too under some conditions.

--
Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com