 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / November 2006A bit confused with SoftReference - when exactly is it collected?
Hello, I'm using a SoftReference to a StringBuffer to build up log messages that are later displayed in a Swing GUI. As this might potentially grow but I don't want the messages to be causing an OOME (as most of them will be written into the log file anyway) I am storing the actual StringBuffer in a SoftReference. My understand was, that a SoftReference will not be collected until the JVM runs out of memory. But it seems that when I run a lengthy task that does create a lot of temporary objects, the soft reference is cleared even though the available memory (max heap) is big enough. The only explanation I have: the JVM will already collect SoftReferences before even expanding the current heap (even if way below the -Xmx value), or - even worse - just at will ;) Can anybody shed some light on this? Currently this does not really help me with my problem. Is there anyway to implement an object that *only* gets collected if the JVM cannot expand the heap any more (-Xmx reached)? Regards Thomas > Hello, > [quoted text clipped - 14 lines] > > Can anybody shed some light on this? The javadocs explain exactly what guarantees are and are not made with regards to SoftReference. As usual for things intimately tied with the garbage collector, a lot of its behaviour is implementation defined. > Currently this does not really help me with my problem. Is there anyway to > implement an object that *only* gets collected if the JVM cannot expand > the heap any more (-Xmx reached)? Not to my knowledge. A simple fix for your design might be to strongly store the last 10 log entries, and then softly store any entries beyond that, which may or may not get reclaimed by the GC, depending on its mood. - Oliver Oliver Wong wrote on 13.11.2006 22:25: > The javadocs explain exactly what guarantees are and are not made with > regards to SoftReference. As usual for things intimately tied with the > garbage collector, a lot of its behaviour is implementation defined. Well the Javadocs says (and that's the reason for my assumptions): "All soft references to softly-reachable objects are guaranteed to have been cleared before the virtual machine throws an OutOfMemoryError. Otherwise no constraints are placed upon the time at which a soft reference will be cleared or the order in which a set of such references to different objects will be cleared." Especially the part about "Otherwise no constraints are placed upon the time", which sounds to me like "The will *only* be collected to prevent an OOME", which clearly isn't the case. > A simple fix for your design might be to strongly > store the last 10 log entries, and then softly store any entries beyond > that, which may or may not get reclaimed by the GC, depending on its mood Yes, I was thinking about that as well... Thomas > Oliver Wong wrote on 13.11.2006 22:25: >> The javadocs explain exactly what guarantees are and are not made [quoted text clipped - 12 lines] > time", which sounds to me like "The will *only* be collected to prevent an > OOME", which clearly isn't the case. No, "no constraints" means the JVM is allowed to do whatever it wants. For example, it could ALWAYS clear the SoftReference immediately, so that the getter always returns null. - Oliver > My understand was, that a SoftReference will not be collected until the > JVM runs out of memory. But it seems that when I run a lengthy task that > does create a lot of temporary objects, the soft reference is cleared > even though the available memory (max heap) is big enough. It is guaranteed that the SoftReference will be cleared in preference to throwing an OOME. The reverse implication is not true. Sun's implementation by default clears a SoftReference if after a collection the number of free megabytes of memory is less that the number of seconds since the reference was last used. The Server HotSpot takes into account the maximum memory available (-Xmx), whereas the Client HotSpot uses the current heap allocation. Tom Hawtin Thomas Hawtin wrote on 13.11.2006 22:52: >> My understand was, that a SoftReference will not be collected until >> the JVM runs out of memory. But it seems that when I run a lengthy [quoted text clipped - 9 lines] > takes into account the maximum memory available (-Xmx), whereas the > Client HotSpot uses the current heap allocation. Thanks for the explanation, especially the difference between client and server VM. Basically what I expected (and have experienced) I do think the Javadocs are a bit vague about this (your statement would have helped me a lot in the Javadocs) Thomas > Sun's implementation by default clears a SoftReference if after a > collection the number of free megabytes of memory is less that the > number of seconds since the reference was last used. The Server HotSpot > takes into account the maximum memory available (-Xmx), whereas the > Client HotSpot uses the current heap allocation. One thing that strikes me with this implementation: what is the difference between a WeakReference and a SoftReference then? I would have expected the described behaviour when using a WeakReference, but not with a SoftReference Thomas > One thing that strikes me with this implementation: what is the > difference between a WeakReference and a SoftReference then? The difference is to allow the JVM room to be more reluctant to clear a SoftReference than a WeakReference. Note that word "allow". I think that in general Oliver's suggestion is the only way to handle this, assuming that you want to use the weak/soft reference architecture at all. But I question whether, for this specific application, soft/weak refs are worth the effort. It would take a fair bit of code without obviously improving the user-experience (either the user wants that info, in which case you shouldn't throw it away, or s/he doesn't, in which case why keep it at all?) You could either write the messages to an ever-growing (temporary?) file, and make the GUI display that, or put a hard limit on how much "history" was ever going to be available, and keep that much unconditionally. -- chris The reference I've never seen much use for is the PhantomReference. Even if you put it in a reference queue, by the time you get the phone call it's gone! I'm not clear where it's useful where a WeakReference won't do as well. On another note, I see mention of the "server VM". I'd always gathered that that is just "what you get when you put -server on the command line", but putting -server on the command line of any of the HotSpot VMs I've gotten downloading Sun's JDKs and JREs produces an error message. Apparently it's actually a separate VM implementation altogether -- and I have no idea where to find it on Sun's site, either. It's not obvious where it is, but it apparently has significant advantages even on the desktop, such as better GC and memory usage, saner SoftReference behavior, and better HotSpot optimization. (Someone profiled some FP code for me and found that FP math in a tight loop runs at C-native speeds with -server but not with a normal Hotspot VM like what I have after there's been time for the JIT to kick in. A normal Hotspot VM is never more than about half C-native speeds at FP math it seems. For scientific applications and simulation work, this is a big freaking deal.) > On another note, I see mention of the "server VM". I'd always gathered > that that is just "what you get when you put -server on the command > line", but putting -server on the command line of any of the HotSpot > VMs I've gotten downloading Sun's JDKs and JREs produces an error > message. java -server works fine for me. It's even mentioned in the commandline help if you only type java Thomas  SignatureIt's not a RootKit - it's a Sony > > On another note, I see mention of the "server VM". I'd always gathered > > that that is just "what you get when you put -server on the command [quoted text clipped - 4 lines] > java -server works fine for me. It's even mentioned in the commandline > help if you only type java Eh. Well, I changed the JRE launch options under Eclipse to add "-server" one time and all attempts to launch something then failed with an error message implying that the switch was invalid for the JVM I was using. Perhaps it's a weird interaction with Eclipse? But nope: I get it at the command line too: Error: no `server' JVM at `C:\PROGRA~1\Java\jre1.6.0\bin\server\jvm.dll'. Same with the 1.5.0_06 JVM, in case anyone's wondering. So, it looks like at minimum you need a DLL that isn't installed by default. (And it looked like the installer installed everything by default! I didn't find any deselected, installable options to check, at any rate, and I specifically looked for such because of exactly this issue when installing 1.6.0.) In fact, there is no subdirectory "server" in any of the JRE directories, only "client" subdirectories... Twisted wrote on 14.11.2006 16:57: > But nope: I get it at the command line too: > [quoted text clipped - 8 lines] > any rate, and I specifically looked for such because of exactly this > issue when installing 1.6.0.) As others have mentioned, the server VM is only available when using the JDK, but you are using a JRE not a JDK! Download the full JDK and point Eclipse to that. Thomas > The reference I've never seen much use for is the PhantomReference. > Even if you put it in a reference queue, by the time you get the phone > call it's gone! I'm not clear where it's useful where a WeakReference > won't do as well. There's two ways of using References. The most obvious is you read the reference and check it hasn't gone. The other way is to subclass and ignore get. The point of subclassing is that when you pick the Reference off the ReferenceQueue relevant information is available (although not the actual target object). PhantomReference can be used for cleanup, much the same as finalisers. Finalisers are, however, easier to write (I say easier, but to get them correct still isn't easy). > On another note, I see mention of the "server VM". I'd always gathered > that that is just "what you get when you put -server on the command [quoted text clipped - 5 lines] > advantages even on the desktop, such as better GC and memory usage, > saner SoftReference behavior, and better HotSpot optimization. Let me guess. You are using Windows. IIRC, the server VM on Windows is in the JDK but not the JRE. However, you can copy it from the JDK into the JRE. There is no AMD64 client VM, but taking the opposite approach from Windows, if you ask for -d64 -client it silently gives the 64-bit server VM instead (silently unless you use -showversion, of course). > (Someone > profiled some FP code for me and found that FP math in a tight loop [quoted text clipped - 3 lines] > math it seems. For scientific applications and simulation work, this is > a big freaking deal.) Yup. Also note trig functions in particular will run faster on non-x86 processors, as the x87 is too imprecise for the Java specification. Tom Hawtin > > The reference I've never seen much use for is the PhantomReference. > > Even if you put it in a reference queue, by the time you get the phone [quoted text clipped - 8 lines] > much the same as finalisers. Finalisers are, however, easier to write (I > say easier, but to get them correct still isn't easy). But what can subclassing PhantomReference do for you that finalizers can't? > Let me guess. You are using Windows. That one's easy. 90% of the population are, so you're going to guess right nine out of ten times if you say that. :) > IIRC, the server VM on Windows is > in the JDK but not the JRE. However, you can copy it from the JDK into > the JRE. Hrm. > There is no AMD64 client VM, but taking the opposite approach from > Windows, if you ask for -d64 -client it silently gives the 64-bit server > VM instead (silently unless you use -showversion, of course). So I might already be getting it then (64-bit dual core Athlon here). Nope. Quick commandline test with the 1.6.0 JRE shows "Client VM" in the version string. Looks like the AMD64 remark is dead wrong, or the installer stupidly installed a generic-x86 VM even though it could peek at my system specs and decide what to download. > > (Someone > > profiled some FP code for me and found that FP math in a tight loop [quoted text clipped - 6 lines] > Yup. Also note trig functions in particular will run faster on non-x86 > processors, as the x87 is too imprecise for the Java specification. Won't this vary modulo -strictfp? (Of course, for some scientific uses, -strictfp is needed for best results, as opposed to speediest results.) >> There's two ways of using References. [...] > > But what can subclassing PhantomReference do for you that finalizers > can't? o You don't have to modify the original class. o The object cannot be resurrected. o You get to run in a thread of your choosing. >> Let me guess. You are using Windows. > > That one's easy. 90% of the population are, so you're going to guess > right nine out of ten times if you say that. :) My guess is that less than 90% of Java programmers do. > So I might already be getting it then (64-bit dual core Athlon here). Not unless you explicitly download it. In general you would download the JRE, then install the 64-bit runtime inside it. If you have it installed you should see a number of amd64 directories including bin/amd64, jre/bin/amd64 and jre/lib/amd64. Installing the 64-bit version may well help performance. (In general moving to 64-bit just means that addresses take up twice the memory and twice memory bandwidth. However, x64 has a less insane instruction set that x86.) > Won't this vary modulo -strictfp? (Of course, for some scientific uses, > -strictfp is needed for best results, as opposed to speediest results.) It goes further than strictfp. Tom Hawtin > >> There's two ways of using References. [...] > > > > But what can subclassing PhantomReference do for you that finalizers > > can't? > > o You don't have to modify the original class. Ah -- so you can use a PhantomReference to make a "finalizer" run when, say, a particular String gets gc'd, or something else that isn't of a class you wrote. > o The object cannot be resurrected. Finalizers can do this? I suppose by putting a strong reference to the object somewhere that is strongly reachable and will still be strongly reachable after the finalizer exits. > o You get to run in a thread of your choosing. As opposed to some system thread you don't want to hold up doing some lengthy crunching; you can poll the queue from the event dispatch thread if you want to change something in a GUI in your "finalizer" without messing around with invokeLater() and generating unreadable code full of anonymous inner classes/cluttering your source directory with still more tiny non-anonymous classes on top of all those little exception classes; etc. > My guess is that less than 90% of Java programmers do. 80% then? :) > Not unless you explicitly download it. In general you would download the > JRE, then install the 64-bit runtime inside it. If you have it installed [quoted text clipped - 5 lines] > twice memory bandwidth. However, x64 has a less insane instruction set > that x86.) And you get these special binaries where, and install them how (if not self-explanatory once you have them)? (The installer didn't do it automatically, and the download links on Sun's site include links for 64-bit binaries for 64-bit Windows, but not 64-bit binaries for 32-bit windows. Although I have e.g. 64-bit binaries for Quake IV and the like here...) > > Won't this vary modulo -strictfp? (Of course, for some scientific uses, > > -strictfp is needed for best results, as opposed to speediest results.) > > It goes further than strictfp. StrictMath then? "Twisted" <twisted0n3@gmail.com> said: >> Not unless you explicitly download it. In general you would download the >> JRE, then install the 64-bit runtime inside it. If you have it installed [quoted text clipped - 12 lines] >windows. Although I have e.g. 64-bit binaries for Quake IV and the like >here...) Go to http://java.sun.com/javase/downloads/index.jsp and choose the ">>Download" link beside "JDK 1.5.0 Update 9". Look through the list; Windows x64 things are at the bottom of the list. Installation instructions are at http://java.sun.com/j2se/1.5.0/install.html . SignatureWolf a.k.a. Juha Laiho Espoo, Finland (GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++ "...cancel my subscription to the resurrection!" (Jim Morrison) > Go to > http://java.sun.com/javase/downloads/index.jsp > > Look through the list; Windows x64 things are at the bottom of the list. jdk-6-rc-windows-amd64.exe is what I came up with. Doesn't Work for Me(tm) -- when the 40-megabyte file is actually finally downloaded and is launched, it simply says my processor architecture is not supported and self-terminates. My processor architecture, of course, being an Athlon 64. And no, I didn't launch the thing on the wrong box (the older AMD32 on my desk is not even connected to anything right now), if that's what you're wondering. (The bumper sticker on the case reads "My *other* other computer is a Cray", in case anyone's wondering.) Now what? I'm not generally *terribly* pleased with recommendations that result in my transferring 40MB of data from a slow Web server for an end result consisting of a grand total of 40 bytes of text -- that's an efficiency of about 0.000001%, in case anyone is wondering that is bad at math... >> Go to >> http://java.sun.com/javase/downloads/index.jsp [quoted text clipped - 8 lines] > older AMD32 on my desk is not even connected to anything right now), if > that's what you're wondering. You *are* running a 64bit version of Windows? > You *are* running a 64bit version of Windows? No, but I *do* have an amd64 processor. It didn't say anything about the OS -- it said specifically that it was my processor architecture that wasn't supported. Either the message is in error, or the whole thing is in error. Also, I *specifically* asked in this group about binaries for Win32 on 64 bit processors, not Win64. >> You *are* running a 64bit version of Windows? > [quoted text clipped - 6 lines] > Also, I *specifically* asked in this group about binaries for Win32 on > 64 bit processors, not Win64. I've lost track of the thread, but in case this information is useful to you: If you're not using Win64, then you need the 32-bit binaries, not the 64 bit ones. For all intents and purposes, if you don't have a 64-bit OS, then all your user-programs will treat your CPU as if it were a 32-bit one - Oliver > I've lost track of the thread, but in case this information is useful to > you: If you're not using Win64, then you need the 32-bit binaries, not the > 64 bit ones. For all intents and purposes, if you don't have a 64-bit OS, > then all your user-programs will treat your CPU as if it were a 32-bit one Really? My experience has been mixed: some 64-bit-enhanced versions of stuff work fine on my system (Quake 4, for one). Others don't (ghostscript, Sun's JVM). Also, it seems strange that my computer would be sold with a 64-bit processor and a 32-bit OS if that made the processor choice senseless without an additional purchase by the customer (of a 64-bit OS -- I assume that you can't just upgrade OEM XP MCE to a 64-bit edition for free, at least not legally)... >> I've lost track of the thread, but in case this information is useful >> to [quoted text clipped - 5 lines] > > Really? I haven't tried first-hand experiments, but that's what I've been told by reliable sources. > My experience has been mixed: some 64-bit-enhanced versions of > stuff work fine on my system (Quake 4, for one). Others don't > (ghostscript, Sun's JVM). Maybe Quake 4 does some fancy CPU detection and changes instruction set depending on the results? I have no idea why it would work. > Also, it seems strange that my computer would be sold with a 64-bit > processor and a 32-bit OS if that made the processor choice senseless > without an additional purchase by the customer (of a 64-bit OS -- I > assume that you can't just upgrade OEM XP MCE to a 64-bit edition for > free, at least not legally)... The 64-bit drivers for hardware haven't matured in terms of optimizations yet, so a lot of vendors install 32-bit Windows, even on a 64-bit processor, to get better benchmarks to show to their potential customers. - Oliver [snip] So this means it's just not likely for a while to get full mileage out of x64 hardware yet? (It's either use a 32 bit OS and some 32 bit apps, or have some driver inefficiencies?) > So this means it's just not likely for a while to get full mileage out > of x64 hardware yet? (It's either use a 32 bit OS and some 32 bit apps, > or have some driver inefficiencies?) As far as I know, correct. - Oliver >> You *are* running a 64bit version of Windows? > [quoted text clipped - 6 lines] > Also, I *specifically* asked in this group about binaries for Win32 on > 64 bit processors, not Win64. If you are running Win32 you will only be able to run Win32 programs and all programs will "think" that you have a 64bit CPI because the CPU will run in 32bit mode. *Only* if you run Win64 you will be able to run programs for the 64bit processors. How do you expect a 64bit program to run on a 32bit OS? But this is *way* off topic a Java newsgroup now. Thomas > On another note, I see mention of the "server VM". I'd always gathered > that that is just "what you get when you put -server on the command > line", but putting -server on the command line of any of the HotSpot > VMs I've gotten downloading Sun's JDKs and JREs produces an error > message. As I recall, you only get the client JVM if you download the standalone JRE; you get both when you download the full development environment (JDK). SignatureMark Jeffcoat Austin, TX > > On another note, I see mention of the "server VM". I'd always gathered > > that that is just "what you get when you put -server on the command [quoted text clipped - 5 lines] > download the standalone JRE; you get both when you > download the full development environment (JDK). I have downloaded full JDKs. (1.5.0_06 and 1.6.0, both currently installed). >> As I recall, you only get the client JVM if you >> download the standalone JRE; you get both when you >> download the full development environment (JDK). > > I have downloaded full JDKs. (1.5.0_06 and 1.6.0, both currently > installed). [from another bit of this thread] > Error: no `server' JVM at > `C:\PROGRA~1\Java\jre1.6.0\bin\server\jvm.dll'. > Same with the 1.5.0_06 JVM, in case anyone's wondering. You may have downloaded it, but you're still running Java from the JRE, not the JDK. Check your path. SignatureMark Jeffcoat Austin, TX Hrm -- that works. But it seems like a strange decision to make -- as if they thought that server software is going to be perpetually in development and never used except by java developers or something. :) Any notion on the 64-bit optimizations under 32-bit Windows? I repeat that I didn't see any binaries (for 1.5.0_0x or 1.6.0) on Sun's site that were 64-bit anything that didn't seem to be for 64-bit Windows... > Hrm -- that works. But it seems like a strange decision to make -- as > if they thought that server software is going to be perpetually in > development and never used except by java developers or something. :) I think it's more along the lines of "People are making fun of Java 'cause you have to download a huge 300MB runtime just to run a 200KB applet. What can we get rid of from the end-user's stuff to make it as small as possible?" - Oliver > > Hrm -- that works. But it seems like a strange decision to make -- as > > if they thought that server software is going to be perpetually in [quoted text clipped - 4 lines] > What can we get rid of from the end-user's stuff to make it as small as > possible?" I don't think the server VM is much bigger than the client VM. It's the JDK that's much bigger than the JRE, but that's becaus WHOA. SEVERE bug in Firefox just now! I was typing away (above) when a download (that x64 file hidden at the bottom of the 1.6.0 page -- for some reason it's at an oddball raw-IP instead of *.sun.com and fetched at less than 1Mb/s -- are you sure it doesn't require 64-bit WinXP Pro?) finished. Fine, but it stole the focus, ate several keypresses, and nearly destroyed this half-edited post in the process. For some reason, something popped up, grabbed the focus, then disappeared. Then this tab scrolled around a bit by itself and disappeared. Well, the form disappeared. It was apparently trying to go to a different site. Of course, the form contents would still be here if I returned from there, because this is Firefox, not Internet Exploder. Except that it seemed specifically to be going to gmail, and it seemed to be doing so because it thought I'd clicked at least 3 times (rather than the actual 0, hands not having been anywhere remotely close to the mouse) on the "back" button. As I'm sure you're well aware, gmail exhibits broken behavior, at least in Firefox, in that if you go "back" to it, the "forward" button greys out. Somehow gmail hijacks your browser and compels it to dump its forward history, perhaps to "punish" people who leave and then return (why though??). Of course, if that had happened, the contents of this form would have been destroyed irrecoverable. This makes the Firefox bug triggered by typing into a form while a download is finishing a Class A Showstopper (severe, difficult to guaranteeably avoid, no known workaround, capable of causing catastropic loss of data without hope of recovery)! I only recovered because I realized what the thing was trying to do in time and hit the forward button several times rapidly before it had loaded anything from gmail. If it had, gmail would have sent whatever code it sends that destroys the browser's forward history almost immediately. Thank Christ gmail's servers are so slow. I never thought I'd be saying that. Jesus. OK, now back to our regular programming. Where was I? Oh yes. I don't think the server VM is much bigger than the client VM. It's the JDK that's much bigger than the JRE, but that's because the JDK contains all the development kit stuff, not because it contains a server VM. >> > Hrm -- that works. But it seems like a strange decision to make -- as >> > if they thought that server software is going to be perpetually in [quoted text clipped - 5 lines] >> What can we get rid of from the end-user's stuff to make it as small as >> possible?" [...] > I don't > think the server VM is much bigger than the client VM. It's the JDK > that's much bigger than the JRE, but that's because the JDK contains > all the development kit stuff, not because it contains a server VM. Every little bit helps (pun intended). - Oliver >> One thing that strikes me with this implementation: what is the >> difference between a WeakReference and a SoftReference then? [quoted text clipped - 11 lines] > GUI display that, or put a hard limit on how much "history" was ever going to > be available, and keep that much unconditionally. As I said, the error messages are also logged into a log file but which cannot be displayed directly as it contains a lot more information. My assumption was: the user will more likely trade a successful completion of the task against losing some informational messages. And for this the SoftReference sounded nice - keep the stuff in memory until we run out of memory. Only then the messages are removed. I guess the approach with the numer of messages is the only sensbile then. Thanks for all the feedback Thomas >> Sun's implementation by default clears a SoftReference if after a >> collection the number of free megabytes of memory is less that the [quoted text clipped - 4 lines] > One thing that strikes me with this implementation: what is the > difference between a WeakReference and a SoftReference then? http://java.sun.com/developer/technicalArticles/ALT/RefObj/ I also find that the javadoc is not vague. Typically you have to read specs very carefully. Kind regards robert >> Sun's implementation by default clears a SoftReference if after a >> collection the number of free megabytes of memory is less that the number [quoted text clipped - 4 lines] > One thing that strikes me with this implementation: what is the difference > between a WeakReference and a SoftReference then? The way I remember it: The referent of a WeakReference does not survive a gc cycle (unless otherwise strongly or softly reachable); The referent of a SoftReference may survive a gc cycle (if the VM can free up enough memory), but is not guaranteed to. The intent is that weakly reachable objects are collected before softly reachable objects. Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.