Java Forum / General / November 2007
finalize() overhead
Joe Seigh - 08 Nov 2007 02:06 GMT I guess overriding finalize() isn't recommended too much because of it's adverse effect on GC performance. But I assume that if it's used a lot. You need finalize() for things like guaranteeing proper clean up of non-memory resources like file descriptors and db connections. One way anyway.
So this sort of use of finalize is considered acceptable? Or should it be avoided at all costs even if you leak file descriptors or whatnot?
 Signature Joe Seigh
When you get lemons, you make lemonade. When you get hardware, you make software.
Lew - 08 Nov 2007 02:17 GMT > I guess overriding finalize() isn't recommended too much because > of it's adverse effect on GC performance. One of a few reasons.
> But I assume that if it's used a lot. What did you mean by this?
> You need finalize() for things like guaranteeing proper clean up of non-memory resources like file descriptors and > db connections. One way anyway. No, you don't. In fact, you pretty much need to avoid finalize() for these purposes.
> So this sort of use of finalize is considered acceptable? Or should No.
> it be avoided at all costs even if you leak file descriptors or > whatnot? Finalizers should not be "avoided at all costs". Nor should you use them to release "file descriptors or whatnot". Or for pretty much anything else, except for when it makes sense. Which is nearly never.
Remember that many finalizable objects will *never* have their finalize() method called. If you are relying on finalize() to free resources you're going to run out of resources. Even if such an object does get its finalize() called, it still takes a couple of GC cycles to get rid of it altogether, which does potentially strain your memory resources.
Garbage collection is for *memory* management, not resource management.
A better way to release resources would be to use a Reference queue to tell when an object is finished, and run a thread to pull objects off the queue and dispose (possibly with a method called dispose()) of their resources.
A typical way to release resources is through the use of try ... finally blocks. Once a resource is allocated, enclose its further use in a try() block whose finally() block releases the resource.
You should avoid overriding finalize() except, a) to release JNI memory when an object is garbage collected, or b) for when it really, really makes sense. Which is nearly never.
 Signature Lew
Eric Sosman - 08 Nov 2007 02:38 GMT > I guess overriding finalize() isn't recommended too much because > of it's adverse effect on GC performance. But I assume that [quoted text clipped - 5 lines] > it be avoided at all costs even if you leak file descriptors or > whatnot? Can't remember who said it (Bloch? Eckel? not sure), but the thing to keep in mind is that finalize() is a creature of the garbage collector, the garbage collector's concern is memory and memory only, so finalize() should be about memory and memory only. Eckel (I'm sure, this time) also suggests a debugging role: If an object becomes garbage without its close() or dispose() or disconnect() method being called, a finalize() can emit a warning message to that effect.
If you need to get rid of file handles, database connections, windows, sessions, sockets, locks on the frammis interface, or other non-memory kinds of things, you should provide a close() or dispose() or terminate() or ahhScrewItAll() method to shut down, release, relinquish, or otherwise sever your association with such entities. The finalize() method is *not* the right vehicle for such things; finalize() is *not* a "destructor."
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Joe Seigh - 08 Nov 2007 03:10 GMT >> I guess overriding finalize() isn't recommended too much because >> of it's adverse effect on GC performance. But I assume that [quoted text clipped - 22 lines] > with such entities. The finalize() method is *not* the right > vehicle for such things; finalize() is *not* a "destructor." The key word here is guarantee. Obviously the proper way is to use a close method but the mere existence of such a method doesn't guarantee its proper use.
It sounds like most of the objection here is aesthetic. Any technical issues (beside the GC not offering any real time guarantees)?
 Signature Joe Seigh
When you get lemons, you make lemonade. When you get hardware, you make software.
Eric Sosman - 08 Nov 2007 03:45 GMT >>> I guess overriding finalize() isn't recommended too much because >>> of it's adverse effect on GC performance. But I assume that [quoted text clipped - 26 lines] > use a close method but the mere existence of such a method doesn't > guarantee its proper use. Enforcing proper use seems beyond the capabilities of an API. Hence Eckel's suggestion to use finalize() as a debugging aid: for an object that *should* be dispose()d, you imlement a finalize() that tests the object's status and whines if the object became garbage with no dispose(). It's not perfect -- you can't be sure all objects will be finalized -- but it may help you catch a few bugs.
> It sounds like most of the objection here is aesthetic. Any > technical issues (beside the GC not offering any real time > guarantees)? Not only does GC offer no real time guarantees, it offers no guarantees at all. The JVM can exit -- usually *does* exit -- with garbage uncollected, hence with garbage un-finalized. If you use finalize() for something important, there is every chance that the important something will never be done at all. (No, not even with runFinalizersOnExit(), although I'm not deeply enough versed in the matter to explain all the whys and wherefores: in this I'm just trusting the word of others.)
Summary: If it needs doing, don't rely on finalize() to do it. Use finalize() only when you're perfectly happy if "it" remains un-done when the JVM exits.
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Lew - 08 Nov 2007 05:22 GMT Joe Seigh wrote:
>> It sounds like most of the objection here is aesthetic. Any >> technical issues (beside the GC not offering any real time >> guarantees)? Not aesthetic at all. Entirely for good design and engineering principles.
> Not only does GC offer no real time guarantees, it offers > no guarantees at all. The JVM can exit -- usually *does* exit -- [quoted text clipped - 4 lines] > versed in the matter to explain all the whys and wherefores: in > this I'm just trusting the word of others.) All these GC calls are defined as hints to the collector, not commands. In a sense, the finalize() method itself is a hint. The only thing you can be sure of is that finalizable objects take much, much longer to create and to destroy than regular objects.
> Summary: If it needs doing, don't rely on finalize() to do it. > Use finalize() only when you're perfectly happy if "it" remains > un-done when the JVM exits. Even then, resist the urge. Finalizers don't come cheap. They don't help like you expect. They can positively harm your code. They hurt memory usage. They complicate maintenance. Other than that, they're mildly useful for rare situations.
 Signature Lew
Joe Seigh - 08 Nov 2007 11:32 GMT > Joe Seigh wrote: >>> It sounds like most of the objection here is aesthetic. Any [quoted text clipped - 25 lines] > memory usage. They complicate maintenance. Other than that, they're > mildly useful for rare situations. I can live with the overhead given it would only affect one class of objects. Especially given using a finalizer would be a lot more straightforward than any alternative.
At the metalevel, there seems to be a curious disconnect here. How are non-memory resources less important than memory resources? If some C++ type showed up and claimed that RAII and explicit deallocation of memory was good enough and GC wasn't necessary, they'd be flamed out of existence. But requiring explicit deallocation of non-memory resources is ok somehow, or at least not an important problem.
 Signature Joe Seigh
When you get lemons, you make lemonade. When you get hardware, you make software.
Lew - 08 Nov 2007 14:30 GMT > I can live with the overhead given it would only affect one class of > objects. > Especially given using a finalizer would be a lot more straightforward than > any alternative. How does "It will not work!" seem straightforward?
> At the metalevel, there seems to be a curious disconnect here. How are > non-memory resources less important than memory resources? If some C++ There is a disconnect here. Who ever made such a claim? No one in this conversation.
This is what's called a "straw man" argument. You hold up a claim that no one has made and prove it false. Good work. So what?
> type showed up and claimed that RAII and explicit deallocation of > memory was good enough and GC wasn't necessary, they'd be flamed out of > existence. But requiring explicit deallocation of non-memory resources > is ok somehow, or at least not an important problem. I don't understand your point here. Who spoke of a problem being not important?
Do not use finalize() to release connections. It will not work.
 Signature Lew
Eric Sosman - 08 Nov 2007 14:47 GMT > [...] > At the metalevel, there seems to be a curious disconnect here. How are [quoted text clipped - 3 lines] > existence. But requiring explicit deallocation of non-memory resources > is ok somehow, or at least not an important problem. There's no implication that non-memory resources are in any way less important. If anything, the implication is the reverse: Non-memory resources can be too important to leave to the mercy of a garbage collector ignorant of their significance.
The language has built-in support for managing memory but lacks support for automatic management of database connections, floating-license token reservations, font caches in the video card, and on and on. Perhaps Java's inventors felt that a general framework for managing such an open-ended list of resources was desirable (I said "perhaps," right?) but intractable.
It could be argued that the destructor is just such a framework. I'm not well versed enough in language design to make a convincing argument either way, and perhaps the argument belongs on an advocacy group anyhow. But for whatever reason, Java does not have automatic destructors, and people who seize on finalize() as a destructor substitute are doing themselves no favors.
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Daniel Pitts - 08 Nov 2007 16:44 GMT >> [...] >> At the metalevel, there seems to be a curious disconnect here. How are [quoted text clipped - 23 lines] > on finalize() as a destructor substitute are doing themselves > no favors. I think that in C++, the destructor is just a framework, although it too might not be called if the process dies. In Java, the framework is try/finally, and that has the same potential problem. However, it is better than using finalize :-)
 Signature Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
Joe Seigh - 09 Nov 2007 01:10 GMT >> [...] >> At the metalevel, there seems to be a curious disconnect here. How are [quoted text clipped - 15 lines] > framework for managing such an open-ended list of resources was > desirable (I said "perhaps," right?) but intractable. They seem to be using it themselves. See java.io.FileInputStream docs and source.
> It could be argued that the destructor is just such a > framework. I'm not well versed enough in language design to [quoted text clipped - 3 lines] > on finalize() as a destructor substitute are doing themselves > no favors. It's not a substitute. I never said anything about using it instead of explicit resource clean up. It would be there for the exact same reason java.io uses it.
 Signature Joe Seigh
When you get lemons, you make lemonade. When you get hardware, you make software.
Lew - 09 Nov 2007 03:37 GMT > It's not a substitute. I never said anything about using it instead > of explicit resource clean up. It would be there for the exact same > reason java.io uses it. Well, you seem hell-bent on using finalize() despite the best advice of several people telling you not to. You will suffer slower performance, higher likelihood of memory issues and greater maintenance effort.
What is the "exact same reason java.io uses it"? Please explain.
More importantly, what is the actual effect of the finalize() methods in those library classes?
Is the net value provided a boon or a detriment?
 Signature Lew
Joe Seigh - 09 Nov 2007 11:30 GMT >> It's not a substitute. I never said anything about using it instead >> of explicit resource clean up. It would be there for the exact same [quoted text clipped - 10 lines] > > Is the net value provided a boon or a detriment? It's not best advice. Your main objections seem to be that I'm thinking of using it on everything which will slow things down, and that I'm going to use it for deterministic destruction. Neither is true.
As to why java.io, jdbc drivers, etc.. use it? Maybe you've never had to support a really large code base, or you have and enjoy getting paged at 3 am about something you can't really do anything about at the moment. Which would you rather support? An application that dies because some combination of events caused it to execute a section of code that leaked resources, or an application that could most likely recover those resources before they ran out completely and issued diagnostic warnings instead. Either you get this or you don't.
 Signature Joe Seigh
When you get lemons, you make lemonade. When you get hardware, you make software.
Lew - 09 Nov 2007 13:41 GMT > As to why java.io, jdbc drivers, etc.. use it? Maybe you've never had to > support a really large code base, or you have and enjoy getting paged at [quoted text clipped - 4 lines] > before they ran out completely and issued diagnostic warnings instead. > Either you get this or you don't. Sure, so using an idiom that increases the chances of bugs is your solution. Go for it.
Go for it with the /ad hominem/ remarks, too.
 Signature Lew
Owen Jacobson - 09 Nov 2007 21:24 GMT > >> It's not a substitute. I never said anything about using it instead > >> of explicit resource clean up. It would be there for the exact same [quoted text clipped - 23 lines] > before they ran out completely and issued diagnostic warnings instead. > Either you get this or you don't. Do you trust the finalizer to release locks for you (or your caller) if you forget to release them yourself? And do you trust your (caller's) code to be in a sane state if a forgotten lock is suddenly released?
Do you trust the finalizer to close sockets for you (or your caller) if you forget to release them yourself? And do you trust your (caller's) code to be in a sane state if a forgotten socket is suddenly closed?
Do you trust the finalizer to close a file handle for you (or your caller) if you forget to release them yourself? And do you trust your (caller's) code to be in a sane state if a forgotten file handle is suddenly closed?
These are the sorts of questions that putting cleanup in the finalizer should be bringing up for you. In my own case, the answer to all of them is "No.", so I don't put cleanup in the finalizer and do not rely on finalizers to catch my own mistakes. Finalizer "cleanup" only changes one broken state (caller did not clean up properly) into another broken state (caller did not clean up properly and may have been unprepared for cleanup); it at best masks bugs and never fixes them.
You can't truly enforce correctness on code that calls your code, and attempting to is painful for both your code and your caller's code. The garbage collector is particularly the wrong tool to try to use to coerce others with, since it's tuned for managing exactly one resource: memory.
Daniel Pitts - 08 Nov 2007 05:12 GMT > The key word here is guarantee. Obviously the proper way is to > use a close method but the mere existence of such a method doesn't [quoted text clipped - 3 lines] > technical issues (beside the GC not offering any real time > guarantees)? Actually, its not an aesthetic at all. The garbage collector treats objects with finalize method differently, to the point where you can end up with more garbage lying around. It can affect performance of the GC to the point that you'll get OOME errors more frequently. Also, if the "this" reference escapes the finalize method, it can interfere with the GC further.
 Signature Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
Roger Lindsjö - 08 Nov 2007 15:35 GMT > The key word here is guarantee. Obviously the proper way is to > use a close method but the mere existence of such a method doesn't > guarantee its proper use. But since finalize() is only guaranteed to run before trying to reuse memory (not when object is made available for GC) you can't be certain that it will run. It will also only run once, even if the object becomes reachable after finalize() is invoked by the GC system.
Anecdote: I have seen problems with this in one of my projects. Our installation team complained that the server ran out of file handles, even if they increased the ulimit. We asked what they did to trigger this and they said "nothing, we just install the application, run our tests and then suddenly we have exceptions and the application has aquired X file handles. Sometimes it releases them again after a while, but then exception comes back after a few more hours." X in this case was close to the setting by ulimit. Initially we were unable to reproduce this but we did find a class releasing a stream without closing it. The fix remove problem.
So, why did they get the error while we never saw it during development/testing? The class leaking the file handle was a monitoring class that got to run every few seconds and hardly created any garbage at all. So we used up file handles faster than memory IFF nothing else ran. Of course that never happened for us, we would either run unit tests or duration tests, but no test just started the application and left it alone for >12 hours (the time it took us to trigger the error once we knew the problem). Since the installation team often let this application alone after installation they triggered the error. The application was robust (apart from causing exceptions ;-) ) that after a while it would run scheduled tasks that caused enough garbage to invoke a GC which in turn caused the file handles to be released.
Now we actually include statistics of file handles during load and idle and verify that we get expected results (which sometimes causes us to change code or change our expectations).
//Roger Lindsjö
Daniel Pitts - 08 Nov 2007 16:46 GMT >> The key word here is guarantee. Obviously the proper way is to >> use a close method but the mere existence of such a method doesn't [quoted text clipped - 4 lines] > that it will run. It will also only run once, even if the object becomes > reachable after finalize() is invoked by the GC system. Not only that, but the GC will put off collecting objects with finalize() for much longer than normal. It has adverse effects on the whole algorithm.
 Signature Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
Owen Jacobson - 08 Nov 2007 15:49 GMT > >> I guess overriding finalize() isn't recommended too much because > >> of it's adverse effect on GC performance. But I assume that [quoted text clipped - 26 lines] > use a close method but the mere existence of such a method doesn't > guarantee its proper use. Neither does finalize().
The JVM is free to put off the collection of any given object for as long as it wants, and to finalize objects in any order it feels like. Objects that have not been collected when the program ends are likely never to be finalized. This is an intrinsic feature of Java's finalization and can't be "worked around"; even the runFinalizersAtExit method doesn't guarantee that finalizers will ever be run (trivially, if the JVM dies of a KILL signal, it never gets a chance to finalize anything, but there are other ways to cause it with Sun's JVM).
Since Java has neither destructors nor C#'s using statement and IDisposable interface, the idiom for using and discarding resources in a controlled, guaranteed way is similar to this:
InputStream source = new FileInputStream ("foo.dat"); // do not put code here, ever. try { readDataFrom (source); } finally { source.close (); }
Using finally like this does guarantee that, regardless of whether or not the code represented by readDataFrom succeeds, the source stream will be closed. The same pattern works for any releaseable resources: explicit locks, database connections, and so on. It's a construct that should be familiar to any Java programmer, and while it's a little verbose, it works rather well and gives the programmer complete control over the lifecycle of the resource.
Wojtek - 08 Nov 2007 15:51 GMT Joe Seigh wrote :
> I guess overriding finalize() isn't recommended too much because > of it's adverse effect on GC performance. But I assume that [quoted text clipped - 5 lines] > it be avoided at all costs even if you leak file descriptors or > whatnot? I once tried closing a DB connection in the finalize().
What I found was that the DB connection was being closed while I was in the middle of retrieving rows from the result set.
In other words, it appeared that the finalize method was being called while the object was still being referenced.
I tested this by removing the close from finalize. This was the ONLY change to the code. Oh yes, this behaviour also happened while stepping through the code in the debugger.
close in finalize: retrieve a random number of rows, but NEVER all the rows, usually about 25-40% of the rows
close in the "body" of the class and called explicitly: 100% every time.
I looked at the results of the tests, scratched my head, and decided that I do not know enough about the GC process. And I vowed NEVER to use finalize again :-)
I think this was Java version 1.4x with Apache 4.x
 Signature Wojtek :-)
Joe Seigh - 09 Nov 2007 01:01 GMT > Joe Seigh wrote : >> I guess overriding finalize() isn't recommended too much because [quoted text clipped - 31 lines] > > I think this was Java version 1.4x with Apache 4.x There was no problem with finalize. What happened is the Statement and ResultSet objects reference the connection internally. You dropped the reference to your enclosing object and it's finalizer closed the connection while it was still being referenced and used by the ResultSet object. You assumed that if your enclosing object wasn't being referenced, that the objects contained in it were no longer being referenced.
 Signature Joe Seigh
When you get lemons, you make lemonade. When you get hardware, you make software.
Wojtek - 09 Nov 2007 15:48 GMT Joe Seigh wrote :
> There was no problem with finalize. What happened is the Statement and > ResultSet objects reference the connection internally. You dropped the > reference to your enclosing object and it's finalizer closed the connection > while it was still being referenced and used by the ResultSet object. > You assumed that if your enclosing object wasn't being referenced, that > the objects contained in it were no longer being referenced. In the following I left out all the try/catch/finally for brevity.
Note to new-comers. DO NOT use this pattern. It is flawed and you may end up with many connections left open due to when the GC calls finalize!
public class SQL { private Connection conn;
public SQL() { conn = Databasae.getConnection(); }
public Data getData() { Statement st; ResultSet rs; Data data = new Data();
st = conn.createStatement(); rs = st.executeQuery("select id,name from table");
while ( rs.next() ) { // code to fill data }
return data; }
public viod finalize() { conn.close(); } }
and to use this:
------------------------------- SQL sql = new SQL(); Data data = sql.getData(); sql = null; -------------------------------
So from my thinking: - a connection is established in the constructor - a method retrieves information - when sql is set tu null, the only reference goes away. - some time in the future the GC calls the finalize method.
In practice, the connection appeared to be closed before the "while ( rs.next() )" loop completed.
When I made the following change, the problem went away.
public class SQL { public Data getData() { Connection conn; Statement st; ResultSet rs; Data data = new Data();
conn = Databasae.getConnection(); st = conn.createStatement(); rs = st.executeQuery("select id,name from table");
while ( rs.next() ) { // code to fill data }
conn.close();
return data; } }
I went back and forth several times between the to ways, and the results were consistent.
 Signature Wojtek :-)
Roger Lindsjö - 09 Nov 2007 19:38 GMT > Joe Seigh wrote : > [quoted text clipped - 18 lines] > { > conn = Databasae.getConnection(); I'm a bit qurious about this. What class is Databasae? Is it a pool, a single connection (a pool of one ;-) or does it generate a new connection for each request?
If it returns the same connection each time, then this scenario could happen:
You create SQL(instance 1) which runs to completion. Later you create SQL(instance 2) which gets a large ResultSet. Both instances now have a reference to the same connection. Iteration over the ResultSet causes some garbage so the GC decides to clean up memory. It finds SQL(instance 1) and runs finalize() on it which cases the connection to be closed. Bad news for SQL(instance 2).
//Roger Lindsjö
Wojtek - 09 Nov 2007 20:34 GMT Roger Lindsjö wrote :
>> Joe Seigh wrote : >> [quoted text clipped - 22 lines] > connection (a pool of one ;-) or does it generate a new connection for each > request? It is a pool manager called proxool, Database is a wrapper class.
 Signature Wojtek :-)
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|