 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / July 2008Controlling SoftReferences
I'm implementing a memory-sensitive cache using java.lang.ref.SoftReference. I'm wondering if I can control the sequence in which the references are released. Here's the idea: three objects, A, B, and C. My main class holds a hard reference to A. A holds a soft reference to B, and B holds a soft reference to C. I never want B released unless C is also released. If I have C hold a hard reference to B, will that achieve it? Here's the actual application: I'm writing a btree. I'd like to cache as much as possible in memory, particularly upper-level parent nodes. I can't use hard references, though, because we may have to operate in low and unpredictable memory conditions. So A is the root node, B is a non-leaf node, and C is a leaf node. I want leaf nodes released before parent nodes, both for performance reasons and because navigating the tree is easier if a node's parent is always available. Or maybe there's a better way to do this altogether. Any ideas welcome. > I'm implementing a memory-sensitive cache using > java.lang.ref.SoftReference. I'm wondering if I can control the [quoted text clipped - 8 lines] > a > hard reference to B, will that achieve it? Yes. > Here's the actual application: I'm writing a btree. I'd like to > cache [quoted text clipped - 6 lines] > navigating the tree is easier if a node's parent is always > available. I'd consider keeping the entire tree in memory, using nodes that are as small as possible, ideally just pointers to its children and a SoftReference to all the other information it represents. > I'm implementing a memory-sensitive cache using > java.lang.ref.SoftReference. I'm wondering if I can control the sequence [quoted text clipped - 16 lines] > > Or maybe there's a better way to do this altogether. Any ideas welcome. Can you control which JVM is used? Sun's is an LRU model that will probably do what you want. I would never allow a program to get into the state where a partial cache must work. Sooner or later you'll hit a usage pattern where memory caching yields no hits and the program will die. I'd change the system requirements such that there is guaranteed to be enough memory or a fast enough disk/database to always work. Partial caching is valuable for reducing average latency but you don't want to bet the program's survival on it. Back in the Java 1.0 days, I created a tree of a data objects that implemented their own virtual memory. It had states to represent available memory: Lazy-load memory-only caching, memory caching with asynchronous flush to disk, and disk-only caching. The first two states were guestimated. The last state was implemented by catching OutOfMemoryError, rolling back a transaction, changing states, and resuming. Rolling back a change after running out of memory was hard to do! This tree of 40000 objects had to run in 90MB of memory and support about 100 concurrent read/write transactions at once. Multithreading never seemed hard after getting that working.  SignatureI will not see your reply if you use Google. > Can you control which JVM is used? Sun's is an LRU model that will > probably do what you want. Thanks, good suggestions. Do you have a reference for this? A little googling didn't turn it up. If Sun clears softreferences in a lru sequence then my worries are over. > > Can you control which JVM is used? Sun's is an LRU model that will > > probably do what you want. [quoted text clipped - 3 lines] > Do you have a reference for this? A little googling didn't turn it up. > If Sun clears softreferences in a lru sequence then my worries are over. http://blogs.sun.com/watt/resource/jvm-options-list.html -XX:SoftRefLRUPolicyMSPerMB=<value> Sun uses a very broken method of managing SoftReferences. You should know about their dirty secret if you're using them a for a significant portion of the total heap size. I've seen applications spend 99.99% of their CPU time in GC with in a 14 GB heap because, by default, SoftReferences were locked into memory for at least 4 hours or until the passing of several consecutive full GC failures. Several consecutive full GC failures doesn't happen quickly on a 14 GB heap. SignatureI will not see your reply if you use Google. >>> Can you control which JVM is used? Sun's is an LRU model that will >>> probably do what you want. [quoted text clipped - 15 lines] > passing of several consecutive full GC failures. Several consecutive > full GC failures doesn't happen quickly on a 14 GB heap. Yes, this is extremely broken. So much so that soft refs may not be useable. Thanks for the pointer -- we're going to have to rethink the whole cache. > http://blogs.sun.com/watt/resource/jvm-options-list.html > [quoted text clipped - 8 lines] > Several consecutive full GC failures doesn't happen quickly on a 14 > GB heap. Could you please elaborate on that, or rather, provide some papers? I distinctively remember testing code that depended on SoftReferences being released, and finding they were indeed properly released pretty quickly, maybe after prompting a GC'ion and a little yielding. I don't wish to call your claims wrong, but it would seem to me, given my experience, that they cannot be valid for each and every configuration and/or situation. SignatureDF. to reply privately, change the top-level domain in the FROM address from "invalid" to "net" > > http://blogs.sun.com/watt/resource/jvm-options-list.html > > [quoted text clipped - 16 lines] > experience, that they cannot be valid for each and every configuration > and/or situation. I was curious about this, too. Apparently, it's more of a problem for a server JVM. SoftRefLRUPolicyMSPerMB is an "integer values representing milliseconds per MB of free memory." The Sun HotSpot FAQ says, "The Java HotSpot Server VM uses the maximum possible heap size (as set with the -Xmx option) to calculate free space remaining." 14,000 MB * 1 ms/MB / 3600 s/h ~= 4 h <http://java.sun.com/docs/hotspot/HotSpotFAQ.html> SignatureJohn B. Matthews trashgod at gmail dot com home dot woh dot rr dot com slash jbmatthews >>> http://blogs.sun.com/watt/resource/jvm-options-list.html >>> [quoted text clipped - 25 lines] > > <http://java.sun.com/docs/hotspot/HotSpotFAQ.html> Yeah, I only managed to make the connection between the "MSPerMB" in the switch's name and the aforementioned 14 GB heap after I posted. I am also to blame for not perusing the link Kevin provided carefully enough. Quote: > Starting with Java HotSpot VM implementations in J2SE 1.3.1, softly > reachable objects will remain alive for some amount of time after the > last time they were referenced. Such a statement, that they would indeed be locked on purpose, was what I was looking for. The personal experience I referred to was probably due to either working -client or with more humble amounts of heap. -.- I'm wondering, though, how come only SoftReferences seem to be given such special treatment. Leaving PhantomReferences aside, wouldn't the described GC policy imply the eventuality that WeakReferences might be collected more often than SoftReferences? Shouldn't it be the other way around (or at most at the same rate), in principle? SignatureDF. to reply privately, change the top-level domain in the FROM address from "invalid" to "net" >>>> http://blogs.sun.com/watt/resource/jvm-options-list.html >>>> [quoted text clipped - 23 lines] > described GC policy imply the eventuality that WeakReferences might be > collected more often than SoftReferences? Yes, i think so. > Shouldn't it be the other way around (or at most at the same rate), in > principle? No, i don't think so. I thought the idea of a weak ref is that it gets cleared and collected as soon as it becomes weakly reachable: it's purely a mechanism for detecting objects dying, for purposes of cleanup and managing canonicalization mappings (so i am told). Soft refs, on the other hand, are there for cacheing: they're supposed to last as long as possible after they become softly reachable, in case you want the thing they point to again. Although this lasting a fixed period of time, regardless of memory pressure, thing is a bit mad. tom SignatureWe must perform a quirkafleeg > >>>> http://blogs.sun.com/watt/resource/jvm-options-list.html > >>>> [quoted text clipped - 39 lines] > Although this lasting a fixed period of time, regardless of memory > pressure, thing is a bit mad. Is it really fixed? I got the impression that the ms/MB setting applied to free space: the free space in the maximum possible heap on a server and the free space in the current heap on a client. I'd expect the ttl to fall as existing heap is consumed. SignatureJohn B. Matthews trashgod at gmail dot com home dot woh dot rr dot com slash jbmatthews >>>>>> http://blogs.sun.com/watt/resource/jvm-options-list.html >>>>>> [quoted text clipped - 26 lines] > and the free space in the current heap on a client. I'd expect the ttl > to fall as existing heap is consumed. DOH! You're absolutely right. I didn't read that description properly - my fault, i'm sorry. Now, the description doesn't quite fit with Kevin's reported problem - if the system was doing a lot of GC, that implies that it was out of free space, which should mean that the softref lifetime would be very short. How does the JVM measure time of last use for references? The last time get() was called? So access to an object via hard refs won't affect any soft refs which also point to it? tom SignatureWhen you mentioned INSERT-MIND-INPUT ... did they look at you like this? > >>>>>> http://blogs.sun.com/watt/resource/jvm-options-list.html > >>>>>> [quoted text clipped - 29 lines] > DOH! You're absolutely right. I didn't read that description properly - my > fault, i'm sorry. Not at all. The SoftRefLRUPolicyMSPerMB setting was new to me, and I was intrigued by the different meaning on a server JVM. It's one more thing that has to be identical in the test environment. > Now, the description doesn't quite fit with Kevin's reported problem - if > the system was doing a lot of GC, that implies that it was out of free > space, which should mean that the softref lifetime would be very short. Dunno. I'm guessing that the long initial ttl keeps many SoftReferences around as the heap grows from initial to maximum size. The FAQ suggests a "... general tendency ... for the Server VM to grow the heap rather than flush soft references..." After the heap has grown to maximum size and free space falls, the ttl should fall too. If it takes several full passes to age out all the eligible SoftReferences, that might explain Kevin's observation. Sadly, I've never had such a luxurious heap. :-) > How does the JVM measure time of last use for references? The last time > get() was called? So access to an object via hard refs won't affect any > soft refs which also point to it? I'm not sure I understand. The api says, "Suppose ... the garbage collector determines ... that an object is softly reachable. At that time it may choose to clear atomically all soft references to that object and all soft references to any other softly-reachable objects from which that object is reachable through a chain of strong references." I'm guessing that SoftRefLRUPolicyMSPerMB affects the "may choose to clear" decision, but the "reachable through a chain of strong references" still applies. SignatureJohn B. Matthews trashgod at gmail dot com home dot woh dot rr dot com slash jbmatthews > >>>>>> http://blogs.sun.com/watt/resource/jvm-options-list.html > >>>>>> [quoted text clipped - 39 lines] > > tom The case I had was with a code analyzer. The JVM would spend several minutes doing 6 full GCs in a row, free some memory, run for 10 seconds, then perform another 6 full GCs in a row, and repeat. Setting SoftRefLRUPolicyMSPerMB to something like 2 to 5 allowed it to run smoothly with a nearly full heap. The analyzer was single-threaded on a 4-core system so I used the CMS collector. SignatureI will not see your reply if you use Google. > ... I thought the idea of a weak ref is that it gets > cleared and collected as soon as it becomes weakly reachable: it's [quoted text clipped - 8 lines] > > tom "All soft references to softly-reachable objects are guaranteed to have been cleared before the virtual machine throws an OutOfMemoryError." So I suppose if memory is really tight the fixed period of time may not be enforced. I also wonder why Java provides WeahHashMap but not SoftHashMap? Seems like it would make creating caches easier. -Wayne >> ... I thought the idea of a weak ref is that it gets >> cleared and collected as soon as it becomes weakly reachable: it's [quoted text clipped - 18 lines] > I also wonder why Java provides WeahHashMap but not SoftHashMap? > Seems like it would make creating caches easier. In a WeakHashMap it's the keys that are weak references, not the values. You can build a cache from a normal hashmap by mapping normal objects to SoftReferences. >> I also wonder why Java provides WeahHashMap but not SoftHashMap? >> Seems like it would make creating caches easier. > > In a WeakHashMap it's the keys that are weak references, not the values. > You can build a cache from a normal hashmap by mapping normal objects to > SoftReferences. True, but it would be nice to have this already done in the standard library. Bear in mind that as well as doing the boxing and unboxing, the SoftHashMap should also deal with soft refs getting cleared (detected via a ReferenceQueue or a null return from get()) by removing their entries from the map. That's more of a pain to reliably get right. tom SignatureWhen you mentioned INSERT-MIND-INPUT ... did they look at you like this? >>> I also wonder why Java provides WeahHashMap but not SoftHashMap? >>> Seems like it would make creating caches easier. [quoted text clipped - 9 lines] > removing their entries from the map. That's more of a pain to > reliably get right. Two quibbles: 1. Assuming you constructed that class, it would be misleading to call it SoftHashMap, since it would be very different from a WeakHashMap. 2.. You might not want to remove an entry when value.get() returns null. It's quite possible that the value is a subclass of SoftReference that contains the information needed to recreate the object it refers to. >>>> I also wonder why Java provides WeahHashMap but not SoftHashMap? >>>> Seems like it would make creating caches easier. [quoted text clipped - 14 lines] > 1. Assuming you constructed that class, it would be misleading to call > it SoftHashMap, since it would be very different from a WeakHashMap. It is. I'm not convinced that it would be misleading in that way, though - neither name really tells you what the class does. It would probably be better to call the soft version a CacheMap, and the weak version a CanonicalisationMap, or something like that. I note that a WeakHashMap doesn't weakly reference its values, and so there's a gotcha in its use, as noted in the documentation, where if a value holds a strong reference to its key, the key will never die. Would it have been better if the values were also weakly referenced? You'd have to arrange it so that the key had a strong reference to the value, to prevent values being lost before the corresponding key died. Or i suppose you could use a weak reference to an Entry, which held both key and value. > 2.. You might not want to remove an entry when value.get() returns null. > It's quite possible that the value is a subclass of SoftReference that > contains the information needed to recreate the object it refers to. True. And you could have a library class for that too - define an interface: public interface Recreatable<T> { public Recreator<T> getRecreator() ; } public interface Recreator<T> { public T recreate() ; } (i don't know if there's a way to express using generics that a Recreatable class should return a Recreator for itself, rather than some other recreator; i can't think of one) Require that the values are recreatable, then extract a recreator and stash it along with the value in the entry. Use it to recreate when necessary. Have configurable policies about when to recreate and when to delete. I think this is what we used to call the Memento pattern. Alternatively, do recreation by key, and have an interface: public interface Recreator<K, V> { public V recreate(K key) ; } An instance of which you give to the map on construction. Even if you didn't do any of this, and did the recreation manually, like this: MyThing getThing(MyKey key) { MyThing t = cache.get(key) ; if (t == null) { t = loadThing(key) ; cache.put(key, t) ; } return t ; } Having the SoftCacheMap already written would make life a lot easier. tom Signaturenot to act in accordance with reason is contrary to God's nature -- Pope Benedict XVI > True. And you could have a library class for that too - define an > interface: [quoted text clipped - 10 lines] > Recreatable class should return a Recreator for itself, rather than some > other recreator; i can't think of one) Perhaps (my generics skills are still developing): public interface Recreatable<T> { public Recreator<T extends Recreatable<T>> getRecreator(); } SignatureLew >> True. And you could have a library class for that too - define an >> interface: [quoted text clipped - 17 lines] > public Recreator<T extends Recreatable<T>> getRecreator(); > } Ooh, maybe just: public interface Recreatable<T extends Recreatable<T>> ? tom Signature20 Minutes into the Future Tom Anderson schreef: |>>>> I also wonder why Java provides WeahHashMap but not SoftHashMap? |>>>> Seems like it would make creating caches easier. [quoted text clipped - 33 lines] |> SoftReference that contains the information needed to recreate the |> object it refers to. | Having the SoftCacheMap already written would make life a lot easier. I seem to remember Jakarta Commons Collections offers something like this. It’s called ReferenceMap and you can decide on the kind of references to be used at construction time, for both key and value: http://commons.apache.org/collections/api-release/org/apache/commons/collections /map/ReferenceMap.html ~ I had a hell of a time trying to make it generic, but it works. (Time to repeat once again: the Jakarta people still don’t do generics, but I made a generic version once. It isn’t up to date with the newest code, I’m afraid, but ask me if you’re interested.) H. - -- Hendrik Maryns http://tcl.sfs.uni-tuebingen.de/~hendrik/ ================== http://aouw.org Ask smart questions, get good answers: http://www.catb.org/~esr/faqs/smart-questions.html Marty Powers wrote: > True. And you could have a library class for that too - define an > interface: [quoted text clipped - 10 lines] > Recreatable class should return a Recreator for itself, rather than some > other recreator; i can't think of one) Perhaps (my generics roots are still imbeding): public interface Recreatable<T> { public Recreator<T equates Recreatable<T>> getRecreator(); } SignatureLew - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Ronald Reagan and his ideas about "freedom fighters": "These Islamic fighters in a faraway land have given new meaning to the words 'courage,' 'determination,' and 'strength.' They have set the standard for those who value freedom and independence everywhere in the world... Let all of us who live in lands of freedom, along with those who dream of doing so, take inspiration from the spirit and courage of the Afghan patriots. Let us resolve that their quest for freedom will prevail, and that Afghanistan will become, once again, an independent member of the family of nations." --- Ronald Reagan STATEMENT ON THE 4TH ANNIVERSARY OF THE SOVIET INVASION OF AFGHANISTAN December 27, 1983 <http://www.reagan.utexas.edu/resource/speeches/1983/122783b.htm [...] > I'm wondering, though, how come only SoftReferences seem to be given > such special treatment. Leaving PhantomReferences aside, wouldn't the > described GC policy imply the eventuality that WeakReferences might be > collected more often than SoftReferences? Shouldn't it be the other way > around (or at most at the same rate), in principle? My experience is limited, but I would expect WeakReferences to be collected more eagerly. I had imagined using WeakReferences to implement a cache and SoftReferences to memoize intermediate results that cost more to recover. I _want_ the oldest stuff in the cache to go first. SignatureJohn B. Matthews trashgod at gmail dot com home dot woh dot rr dot com slash jbmatthews - snip - > I'm wondering, though, how come only SoftReferences seem to be given > such special treatment. Leaving PhantomReferences aside, wouldn't the > described GC policy imply the eventuality that WeakReferences might be > collected more often than SoftReferences? Shouldn't it be the other way > around (or at most at the same rate), in principle? WeakReferences and PhantomReferences have no impact on garbage collection of the referred object. When the collection happens depends on which heap the referred object resides in and the frequency of its collection. It could be milliseconds or many hours after the last hard and soft reference is gone. SignatureI will not see your reply if you use Google. > I'm wondering, though, how come only SoftReferences seem to be given > such special treatment. Leaving PhantomReferences aside, wouldn't the > described GC policy imply the eventuality that WeakReferences might be > collected more often than SoftReferences? Shouldn't it be the other way > around (or at most at the same rate), in principle? Ah, shoot. My mistake again. It pays to read the docs again from time to time... The order of references, from strongest to weakest, is: /strong/, /soft/, /weak/, /phantom/ whereas I had laboured under the misapprehension it was in fact: /strong/, /weak/, /soft/, /phantom/ Consequently, the GC policy that has been discussed here isn't particularly nonsensical. Sorry for stirring the mud. SignatureDF. to reply privately, change the top-level domain in the FROM address from "invalid" to "net" >> I'm wondering, though, how come only SoftReferences seem to be given >> such special treatment. Leaving PhantomReferences aside, wouldn't the [quoted text clipped - 13 lines] > Consequently, the GC policy that has been discussed here isn't > particularly nonsensical. Sorry for stirring the mud. Speaking of which: if anyone can explain what a PhantomReference can be used for, I will be in your debt. On 2008-07-09 00:22 +0100, Mike Schilling allegedly wrote: >> The order of references, from strongest to weakest, is: /strong/, >> /soft/, /weak/, /phantom/ [quoted text clipped - 9 lines] > if anyone can explain what a PhantomReference can be used for, I will > be in your debt. Quoting from: <http://java.sun.com/javase/6/docs/api/java/lang/ref/package-summary.html> > phantom references are for scheduling pre-mortem cleanup actions in a > more flexible way than is possible with the Java finalization > mechanism > # An object is /softly reachable/ if it is not strongly reachable but > can be reached by traversing a soft reference. [quoted text clipped - 5 lines] > nor weakly reachable, it has been finalized, and some phantom > reference refers to it. IOW, PhantomReferences don't prevent the referred Object from being finalized. SignatureDF. to reply privately, change the top-level domain in the FROM address from "invalid" to "net" > On 2008-07-09 00:22 +0100, Mike Schilling allegedly wrote: >>> The order of references, from strongest to weakest, is: /strong/, [quoted text clipped - 34 lines] > IOW, PhantomReferences don't prevent the referred Object from being > finalized. Right, at the point that the PhantomReference has been delived, the object is only reachable via PhantomReferences, no one can see it (since PhantomReference.get() always returns null), its finalizer, if any, has run, but its memory hasn't been freed. What is interesting about that situation? To ask it another way, since the memory being freed is transparent to everything, what would I need to do before it happens, rather than, as with a WeakReference, afterward? >> Quoting from: >> <http://java.sun.com/javase/6/docs/api/java/lang/ref/package-summary.html> [quoted text clipped - 23 lines] > freed is transparent to everything, what would I need to do before it > happens, rather than, as with a WeakReference, afterward? Well, I can't say for sure. I haven't ever actually used PhantomReferences. But thinking about it, I would say their use would be as a replacement for finalisers. It's not so much the /situation/ you described above which is interesting, it's the /flow/. There need not be any steps between when an Object becomes phantom reachable and when its memory gets reclaimed -- even enqueueing the PhantomReferences, if any, onto the corresponding ReferenceQueue might be done after that. What's more, the enqueueing itself isn't coupled to the process of GC. In other words, it would amount to an asynchronous finalizer, and one that doesn't impede the GC. Now, how would this be significantly different from SoftReferences or WeakReferences? Don't know exactly. I suppose by the fact that with a PhantomReference's referent is /never/ reachable via the Reference Object. Maybe this eases the GC'tor's work a bit. But I too would be eager to hear more about this. SignatureDF. to reply privately, change the top-level domain in the FROM address from "invalid" to "net" >>> Quoting from: >>> <http://java.sun.com/javase/6/docs/api/java/lang/ref/package-summary.html> [quoted text clipped - 45 lines] > might be done after that. What's more, the enqueueing itself isn't > coupled to the process of GC. It is, and that's one of the things that really baffles me. A PhantomReference, unlike the others, isn't cleared before being enqueued, which is to say, the referred-to object remains phantom reachable. In the sequence of events 1. The object becomes phantom reachable. 2. The PhantomReference in enqueued. 3. The PR is delivered 4. The code that received the PR does some stuff. 5. That code clears the PR 6. That code does some other stuff The GC can't occur until after step 5. Why is that useful? It's not true with a Soft or WeakReference, in which the GC clears the reference before enqueueing it, so the GC may have happened before step 2 and step 5 would be a no-op. >> There need not be any steps between when an Object becomes phantom >> reachable and when its memory gets reclaimed -- even enqueueing the [quoted text clipped - 15 lines] > > The GC can't occur until after step 5. Indeed, and considering how it was written balkc on white in the docs and I didn't see it I must conclude that I'm not able to think clearly anymore and will have to get some sleep. See you then. SignatureDF. to reply privately, change the top-level domain in the FROM address from "invalid" to "net" > >> Quoting from: > >> <http://java.sun.com/javase/6/docs/api/java/lang/ref/package-summary.html> [quoted text clipped - 39 lines] > Object. Maybe this eases the GC'tor's work a bit. But I too would be > eager to hear more about this. PhantomReferences let you process the cleanup of external resources at your own pace with no impact on garbage collection. Just extend the PhantomReference class to contain a reference to the external resource. For example, you might put a string in there that can be used to clean up a cache on a remote server when the referred object is gone. You can cleanup in a finalize() method but the environment is highly restricted. Blocking operations (disk, network, JDBC, synchronized blocks, etc.) will cause OutOfMemoryErrors. Allocating anything can fail because you're interrupting part of the process that makes memory available. While finalize() is running, other objects in the class may have their finalize() methods running at the same time in other threads. There's also a risk that you'll make your object live again, but the finalize() method only runs once. SignatureI will not see your reply if you use Google. >>>> Quoting from: >>>> <http://java.sun.com/javase/6/docs/api/java/lang/ref/package-summary.html> [quoted text clipped - 58 lines] > used to clean up a cache on a remote server when the referred object > is gone. Good. Now why does a PhantomReference do this better than a Weak one does? >> PhantomReferences let you process the cleanup of external resources >> at your own pace with no impact on garbage collection. Just extend >> the PhantomReference class to contain a reference to the external >> resource. Now, just that very moment, this PhantomReference started to make sense to me :-) Thanks, Kevin! > Good. Now why does a PhantomReference do this better than a Weak one > does? You create some instance of your overridden PhantomReference, which contains a copy of the direct handle of the external ressource (just like the referenced(monitored) object. When the Object is gc'ed, your PhR-instance gets added to that queue, and by scanning the queue you get the handle of the external ressource, and you can free it. Now for the difference to Weak/Soft-Refs: If you do the same to a Weak/Soft-Ref, then you bear the risk, that some other part of your code may perform a resurrection of the actual object, after you've successfully cleaned up the external ressource. The Phantom-Ref (having been enqueued) guarantees, that the Object is indeed no longer refer'able, and thus cannot possibly be resurrected, ever. Kevin: did I get it right? >>> PhantomReferences let you process the cleanup of external resources >>> at your own pace with no impact on garbage collection. Just extend [quoted text clipped - 14 lines] > and by scanning the queue you get the handle of the external > ressource, and you can free it. OK. Why do you care whether that happens before the object's memory is freed? > Now for the difference to Weak/Soft-Refs: > If you do the same to a Weak/Soft-Ref, then you bear the risk, [quoted text clipped - 3 lines] > the Object is indeed no longer refer'able, and thus cannot possibly > be resurrected, ever. Because PhantomReferences, uniquely, wait until after the finalizer runs? OK, I see that. >> When the Object is gc'ed, your PhR-instance gets added to that queue, >> and by scanning the queue you get the handle of the external >> ressource, and you can free it. > OK. Why do you care whether that happens before the object's memory is > freed? Whether it's memory is actually freed is irrelevant in this context. I goofed again: when I wrote "is gc'd", I really meant: "is found eligible for gc and has already undergone all the stuff as documented in javadoc for PhantomReference..." :-) The point I was actually trying to make is, that until a few days ago I saw no sense in using a PhantomReference, when I cannot query it about the object it "refers". By subclassing it (and adding fields to hold extra information) one can solve this problem (and there may also be other ways to associate extra information with the PhR). I didn't see this until Kevin mentioned it. >>> When the Object is gc'ed, your PhR-instance gets added to that >>> queue, and by scanning the queue you get the handle of the external [quoted text clipped - 4 lines] > > Whether it's memory is actually freed is irrelevant in this context. Of course it is. What I still don't understand is why PhantomReference is explicitly documented as being delivered with the reference uncleared, so that the code to which it is delivered can control precisely when the memory is freed. > I goofed again: when I wrote "is gc'd", I really meant: "is found > eligible for gc and has already undergone all the stuff as documented [quoted text clipped - 7 lines] > one can solve this problem (and there may also be other ways to > associate extra information with the PhR). Oh. That's true of any Reference when it's delivered via a queue. A Soft or Weak one has already been cleared, so get() returns null. A Phanotm hasn't been cleared, but its get() returns null anyway. >> Whether it's memory is actually freed is irrelevant in this context. > Of course it is. What I still don't understand is why PhantomReference is > explicitly documented as being delivered with the reference uncleared, so > that the code to which it is delivered can control precisely when the memory > is freed. Good point. Perhaps, back in 1.2 the developers at Sun needed it to inspect the memory of removed objects - with other tools... One could perhaps read out the private field "referent" of PhR's base class Reference by means of reflection. :-) Mike Schilling wrote: > if anyone can explain what a PhantomReference can be used for, I will be in > your debt. Here's a quick note: "The PhantomReference class is useful only to track the impending collection of the referring object. As such, it can be used to perform pre-mortem cleanup operations. A PhantomReference must be used with the ReferenceQueue class. The ReferenceQueue is required because it serves as the mechanism of notification. When the garbage collector determines an object is phantomly reachable, the PhantomReference object is placed on its ReferenceQueue. The placing of the PhantomReference object on the ReferenceQueue is your notification that the object the PhantomReference object referred to has been finalized and is ready to be collected. This allows you to take action just prior to the object memory being reclaimed. " The full article here: <http://www-128.ibm.com/developerworks/library/j-refs/> I'm still reading that article myself, I'll post up if I find any great insights. Mark Space schreef: | Mike Schilling wrote: | [quoted text clipped - 14 lines] | allows you to take action just prior to the object memory being | reclaimed. " This reminds me of this recent thread about connections being reclaimed in try blocks etc. I can’t find it though, hopefully the person involved there will read this. (I think I blacklisted some people since the thread got out of hand.) Cheers, H. - -- Hendrik Maryns http://tcl.sfs.uni-tuebingen.de/~hendrik/ ================== http://aouw.org Ask smart questions, get good answers: http://www.catb.org/~esr/faqs/smart-questions.html > I'm implementing a memory-sensitive cache using > java.lang.ref.SoftReference. I'm wondering if I can control the sequence > in which the references are released. I'd rather choose the mental model of sub graphs of the object graph. Inside you can use hard refs but you need to make sure that all paths into the subgraph you want to treat as an entity (GC wise) are soft. > Here's the idea: three objects, A, B, and C. My main class holds a hard > reference to A. A holds a soft reference to B, and B holds a soft > reference to C. > > I never want B released unless C is also released. If I have C hold a > hard reference to B, will that achieve it? Yes. > Here's the actual application: I'm writing a btree. I'd like to cache as > much as possible in memory, particularly upper-level parent nodes. I [quoted text clipped - 3 lines] > parent nodes, both for performance reasons and because navigating the > tree is easier if a node's parent is always available. Yep, then make the parent ref a strong ref. This will ensure that parents are always available. Downside is that as long as one leaf is non collectible at least the path to the root stays as well. Kind regards robert > I'm implementing a memory-sensitive cache using > java.lang.ref.SoftReference. I'm wondering if I can control the sequence [quoted text clipped - 16 lines] > > Or maybe there's a better way to do this altogether. Any ideas welcome. I would avoid fancy references at all costs. You have complex enough problem at your hand, and probably don't want to complicate the matter futher. As you are writing b-tree I assume the basic unit of storage is block, not an object. (Block abstraction simplifies concurrency issues as well). Therefore, somewhere you have a map of all the cached blocks and need some policy of freeing the cache on low memory event. Thus the b-tree structure goes out of the picture. If this bean thingy that was already mentioned is unreliable in any way, can't you just write a thread that polls the System object explicitly ho much memory is available? Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.