Java Forum / General / October 2005
Manipulate objects in heap
habib - 21 Oct 2005 08:11 GMT Hi there, I would like to access and manipulate live objects in heap directly in a Java program. Does the Sun JVM or other implementations of JVM allow this via their APIs or I need to edit source code of an implementation and create some APIs for this need? Thank you in advance, Habib
Chris Uppal - 21 Oct 2005 08:32 GMT > I would like to access and manipulate live objects in heap directly in > a Java program. Why ? Just curious.
> Does the Sun JVM or other implementations of JVM allow this via their > APIs Sun, IBM, and BEA definitely do /not/ allow anything of the kind. Of course there's nothing to stop you reverse-engineering the memory layout, and then invoking code via JNI that will then ignore the JNI interfaces and access memory directly. Of course, it's virually certain that if you try that then you'll make an enormous mess...
I think that some of the free/hobbyist/research JVMs allow that kind of thing (in the sense of having a more-or-less public API). But again it comes down to what you want to do. If, just for instance, you want to experiment with alternative GC techniques then the problem is that the "second-tier" JVMs don't tend to have cutting-edge GC implementations of their own, so you have nothing to compare your stuff with.
Still, if I were wanting to play with low-level details, then I'd be tempted to use the IBM "jikes" JVM as a basis (not to be confused with the jikes compiler), simply because it's written in Java, and has some interesting papers published about its design and implementation.
-- chris
habib - 21 Oct 2005 17:11 GMT Thank you so much dear Chris, I surely check the JVM you wrote. I need to change the heap for editing objects, for example, adding a method to a object. It's good idea for changing software without stopping it. Habib
Ross Bamford - 21 Oct 2005 20:43 GMT > Thank you so much dear Chris, > I surely check the JVM you wrote. > I need to change the heap for editing objects, for example, adding a > method to a object. > It's good idea for changing software without stopping it. > Habib Looking at this and your other message, I think I see what you're trying to do - you believe that by somehow modifying the memory allocated to instances you'll be able to modify methods. Perhaps you're thinking of implementing dynamic invocation along the lines of Smalltalk and Ruby?
I think I've as open a mind as anyone with this kind of stuff, but take it from me that's just not possible given the architecture of the JVM. The memory you're hoping to modify has nothing to do (directly) with the methods, and the separation between classes and instances is almost total. It's not possible, with the current architecture, to do anything more than emulate this kind of thing (a-la Groovy, in which nearly every call is reflective, and it shows, or Nice which I believe uses tableswitch to do it), and then it can't be integrated directly with the Java language.
As I said in my reply to your other message on c.l.j.help, the tools are available, but even with all the flexibility in the world I doubt you'll get what you're after. "Changing software without stopping it" does have some attractive points, as you say, but this isn't how it's going be done I think, at least in Java.
(Which is also to say: If you do achieve it, I'd be very interested in how
:))
 Signature Ross Bamford - rosco@roscopeco.remove.co.uk
Ross Bamford - 21 Oct 2005 21:01 GMT >> Thank you so much dear Chris, >> I surely check the JVM you wrote. [quoted text clipped - 7 lines] > instances you'll be able to modify methods. Perhaps you're thinking of > implementing dynamic invocation along the lines of Smalltalk and Ruby? Sorry, missed a question mark out there - I'm not trying to tell you what you believe :)
> you believe that by somehow modifying the memory allocated to instances > you'll be able to modify methods > ? <
 Signature Ross Bamford - rosco@roscopeco.remove.co.uk
Roedy Green - 22 Oct 2005 01:11 GMT >I need to change the heap for editing objects, for example, adding a >method to a object. >It's good idea for changing software without stopping it. The way you do that is to use custom class loaders. Then you can have two different object layouts with different sets of methods, with the same name living simultaneously in the same JVM. Once you convert your old objects to new, or they all die a natural death, you can delete the corresponding class loader and recover all the old method space too.
see http://mindprod.com/jgloss/classloader.html
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Again taking new Java programming contracts.
Chris Uppal - 22 Oct 2005 12:23 GMT > I need to change the heap for editing objects, for example, adding a > method to a object. > It's good idea for changing software without stopping it. If the reason you want to do this is to experiment, to learn, and to play with new concepts then go ahead: have lots of fun! You might even get a research paper out of it (although dynamic modification of class definitions is not itself new -- it has been around since at least the 1980's).
The rest of this reply is on the assumption that that's /not/ the case. If you /need/ to do this -- i.e. you have a design for "real" code (code that will be used in a production environment by real people to help with real problems) then I don't think that's the right way to approach it. You will never be able to make it work reliably if you base it on a production-grade JVM, and you can't use a less-than-production-grade JVM as the basis for a production system.
The "correct" way to allow classes to change definition are:
1) Use classloaders. This will allow several versions of the "same" (same name) class to exist -- at the same time if necessary. The big problem is that you can't change the definition of the class of an existing object -- you can create a new definition of a class, but all the instances of the old class remain just that: instances of the old class (which does not change).
2) The other way is not (IMO) suitable for production code, which is to use the debugging API to change the definition of the running code. That is severely limited, and in any case not supported by all JVMs. But within (mostly undocumented) limits you can modify classes in a running application.
3) The last way is simply to give up on Java. Its designers and the creators of its JVM definition (and the canonical implementation) -- Sun -- simply do not "get" dynamic languages. So there is no real support for dynamic operations. You either have to use a complete layer of interpretation above the JVM stuff (which amounts to using a different language) or you use an explicitly different language which doesn't have a crippled semantics. Personally I prefer Smalltalk, but that is by no means the only languages with the ability to handle dynamic constructs, for instance Ruby and Python are both dynamic.
-- chris
habib - 22 Oct 2005 16:48 GMT First of all, thank you very much for your peirces reply. The ways you wrote are the ways I have so much experience with them. If I want to answer your question about my objective of changing classes, I must say both yes and no! yes because I'm candidate for Master of Science and changing software in runtime is my thesis and no because one of objective of my thesis is that my solution can work in real production problems. About the ways you suggested, I must say that I tried way 1, but I don't like it because of the problem you touched on it. then I wrote a program which works with way 2; The program gets new requirements via a XML language, transforms it to Java source code using a XSLT processor, then invokes javac for compiling it, and finally replaces it with old one using JPDA API. But I search for a better solution, since the only change JPDA allows is modifying body of methods, while I need to change classes in any way, for example adding or removing methods.... In this way, I think the 2 sites Ross sent, can help me. Also, Please write to me, if you have any idea for improving my works. Thanks again, Habib
Chris Uppal - 25 Oct 2005 10:29 GMT > yes because I'm candidate for Master of Science and changing software > in runtime is my thesis and no because one of objective of my thesis is > that my solution can work in real production problems. Hmm, I don't really know what to suggest.
I don't, personally, doubt if it's feasible to allow class definition changes on a /production/ JVM. (After all, the people at Sun would presumably like to remove as many restrictions on JPDA as possible, and if /they/ can't do it...)
It seems to me that you have the choice between modifying a non-production JVM, and saying, in effect, "my work demonstrates how to add these features to a JVM, and shows that it is possible to do so without making the result /less/ suitable for use in a production environment". Or, alternatively, of pursuing an approach based on putting a layer of interpretation and/or indirection between the code that is to be made dynamic and the actual JVM runtime. In the latter case you could either use an interpretive but Java-like language (such as Groovy), or could maybe look into byte-code re-writing to replace hard-wired (in the bytecode) method calls and field accesses with an indirection via datastructures which you control and can modify at runtime.
Best of luck, anyway!
-- chris
Roedy Green - 25 Oct 2005 14:46 GMT >First of all, thank you very much for your peirces reply. >The ways you wrote are the ways I have so much experience with them. >If I want to answer your question about my objective of changing >classes, I must say both yes and no! I can see two ways to approach this.
1. switch languages. Smalltalk is more amenable to this sort of on the fly tinkering.
2. think in terms of creating new classes, copying data from old objects to the new ones in a way that hardly anyone notices the switch. Perhaps in your new language all pointers have to be two way, or you need access to the low level ability to patch pointers, much like what GC can do to patch pointers to a new location.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Again taking new Java programming contracts.
Roedy Green - 22 Oct 2005 01:07 GMT >I would like to access and manipulate live objects in heap directly in >a Java program. Literally answered your question is. there is not much else you CAN do in Java. All objects live on the heap and the various methods manipulate the live object fields.
Perhaps you are using terminology is some way different from the way Java programmers do.
 Signature Canadian Mind Products, Roedy Green. http://mindprod.com Again taking new Java programming contracts.
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|