Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / November 2007

Tip: Looking for answers? Try searching our database.

finalize() overhead

Thread view: 
Joe Seigh - 08 Nov 2007 02:06 GMT
I guess overriding finalize() isn't recommended too much because
of it's adverse effect on GC performance.  But I assume that
if it's used a lot.  You need finalize() for things like guaranteeing
proper clean up of non-memory resources like file descriptors and
db connections.  One way anyway.

So this sort of use of finalize is considered acceptable?  Or should
it be avoided at all costs even if you leak file descriptors or
whatnot?

Signature

Joe Seigh

When you get lemons, you make lemonade.
When you get hardware, you make software.

Lew - 08 Nov 2007 02:17 GMT
> I guess overriding finalize() isn't recommended too much because
> of it's adverse effect on GC performance.

One of a few reasons.

> But I assume that if it's used a lot.

What did you mean by this?

> You need finalize() for things like guaranteeing proper clean up of non-memory resources like file descriptors and
> db connections.  One way anyway.

No, you don't.  In fact, you pretty much need to avoid finalize() for these
purposes.

> So this sort of use of finalize is considered acceptable?  Or should

No.

> it be avoided at all costs even if you leak file descriptors or
> whatnot?

Finalizers should not be "avoided at all costs".  Nor should you use them to
release "file descriptors or whatnot".  Or for pretty much anything else,
except for when it makes sense.  Which is nearly never.

Remember that many finalizable objects will *never* have their finalize()
method called.  If you are relying on finalize() to free resources you're
going to run out of resources.  Even if such an object does get its finalize()
called, it still takes a couple of GC cycles to get rid of it altogether,
which does potentially strain your memory resources.

Garbage collection is for *memory* management, not resource management.

A better way to release resources would be to use a Reference queue to tell
when an object is finished, and run a thread to pull objects off the queue and
dispose (possibly with a method called dispose()) of their resources.

A typical way to release resources is through the use of try ... finally
blocks.  Once a resource is allocated, enclose its further use in a try()
block whose finally() block releases the resource.

You should avoid overriding finalize() except, a) to release JNI memory when
an object is garbage collected, or b) for when it really, really makes sense.
 Which is nearly never.

Signature

Lew

Eric Sosman - 08 Nov 2007 02:38 GMT
> I guess overriding finalize() isn't recommended too much because
> of it's adverse effect on GC performance.  But I assume that
[quoted text clipped - 5 lines]
> it be avoided at all costs even if you leak file descriptors or
> whatnot?

    Can't remember who said it (Bloch?  Eckel?  not sure), but
the thing to keep in mind is that finalize() is a creature of
the garbage collector, the garbage collector's concern is
memory and memory only, so finalize() should be about memory
and memory only.  Eckel (I'm sure, this time) also suggests
a debugging role: If an object becomes garbage without its
close() or dispose() or disconnect() method being called, a
finalize() can emit a warning message to that effect.

    If you need to get rid of file handles, database connections,
windows, sessions, sockets, locks on the frammis interface, or
other non-memory kinds of things, you should provide a close()
or dispose() or terminate() or ahhScrewItAll() method to shut
down, release, relinquish, or otherwise sever your association
with such entities.  The finalize() method is *not* the right
vehicle for such things; finalize() is *not* a "destructor."

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Joe Seigh - 08 Nov 2007 03:10 GMT
>> I guess overriding finalize() isn't recommended too much because
>> of it's adverse effect on GC performance.  But I assume that
[quoted text clipped - 22 lines]
> with such entities.  The finalize() method is *not* the right
> vehicle for such things; finalize() is *not* a "destructor."

The key word here is guarantee.  Obviously the proper way is to
use a close method but the mere existence of such a method doesn't
guarantee its proper use.

It sounds like most of the objection here is aesthetic.  Any
technical issues (beside the GC not offering any real time
guarantees)?

Signature

Joe Seigh

When you get lemons, you make lemonade.
When you get hardware, you make software.

Eric Sosman - 08 Nov 2007 03:45 GMT
>>> I guess overriding finalize() isn't recommended too much because
>>> of it's adverse effect on GC performance.  But I assume that
[quoted text clipped - 26 lines]
> use a close method but the mere existence of such a method doesn't
> guarantee its proper use.

    Enforcing proper use seems beyond the capabilities of an
API.  Hence Eckel's suggestion to use finalize() as a debugging
aid: for an object that *should* be dispose()d, you imlement
a finalize() that tests the object's status and whines if the
object became garbage with no dispose().  It's not perfect --
you can't be sure all objects will be finalized -- but it may
help you catch a few bugs.

> It sounds like most of the objection here is aesthetic.  Any
> technical issues (beside the GC not offering any real time
> guarantees)?

    Not only does GC offer no real time guarantees, it offers
no guarantees at all.  The JVM can exit -- usually *does* exit --
with garbage uncollected, hence with garbage un-finalized.  If
you use finalize() for something important, there is every chance
that the important something will never be done at all.  (No, not
even with runFinalizersOnExit(), although I'm not deeply enough
versed in the matter to explain all the whys and wherefores: in
this I'm just trusting the word of others.)

    Summary: If it needs doing, don't rely on finalize() to do it.
Use finalize() only when you're perfectly happy if "it" remains
un-done when the JVM exits.

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Lew - 08 Nov 2007 05:22 GMT
Joe Seigh wrote:
>> It sounds like most of the objection here is aesthetic.  Any
>> technical issues (beside the GC not offering any real time
>> guarantees)?

Not aesthetic at all.  Entirely for good design and engineering principles.

>     Not only does GC offer no real time guarantees, it offers
> no guarantees at all.  The JVM can exit -- usually *does* exit --
[quoted text clipped - 4 lines]
> versed in the matter to explain all the whys and wherefores: in
> this I'm just trusting the word of others.)

All these GC calls are defined as hints to the collector, not commands.  In a
sense, the finalize() method itself is a hint.  The only thing you can be sure
of is that finalizable objects take much, much longer to create and to destroy
than regular objects.

>     Summary: If it needs doing, don't rely on finalize() to do it.
> Use finalize() only when you're perfectly happy if "it" remains
> un-done when the JVM exits.

Even then, resist the urge.  Finalizers don't come cheap.  They don't help
like you expect.  They can positively harm your code.  They hurt memory usage.
 They complicate maintenance.  Other than that, they're mildly useful for
rare situations.

Signature

Lew

Joe Seigh - 08 Nov 2007 11:32 GMT
> Joe Seigh wrote:
>>> It sounds like most of the objection here is aesthetic.  Any
[quoted text clipped - 25 lines]
> memory usage.  They complicate maintenance.  Other than that, they're
> mildly useful for rare situations.

I can live with the overhead given it would only affect one class of objects.
Especially given using a finalizer would be a lot more straightforward than
any alternative.

At the metalevel, there seems to be a curious disconnect here.  How are
non-memory resources less important than memory resources?  If some C++
type showed up and claimed that RAII and explicit deallocation of
memory was good enough and GC wasn't necessary, they'd be flamed out of
existence.  But requiring explicit deallocation of non-memory resources
is ok somehow, or at least not an important problem.

Signature

Joe Seigh

When you get lemons, you make lemonade.
When you get hardware, you make software.

Lew - 08 Nov 2007 14:30 GMT
> I can live with the overhead given it would only affect one class of
> objects.
> Especially given using a finalizer would be a lot more straightforward than
> any alternative.

How does "It will not work!" seem straightforward?

> At the metalevel, there seems to be a curious disconnect here.  How are
> non-memory resources less important than memory resources?  If some C++

There is a disconnect here.  Who ever made such a claim?  No one in this
conversation.

This is what's called a "straw man" argument.  You hold up a claim that no one
has made and prove it false.  Good work.  So what?

> type showed up and claimed that RAII and explicit deallocation of
> memory was good enough and GC wasn't necessary, they'd be flamed out of
> existence.  But requiring explicit deallocation of non-memory resources
> is ok somehow, or at least not an important problem.

I don't understand your point here.  Who spoke of a problem being not important?

Do not use finalize() to release connections.  It will not work.

Signature

Lew

Eric Sosman - 08 Nov 2007 14:47 GMT
> [...]
> At the metalevel, there seems to be a curious disconnect here.  How are
[quoted text clipped - 3 lines]
> existence.  But requiring explicit deallocation of non-memory resources
> is ok somehow, or at least not an important problem.

    There's no implication that non-memory resources are in
any way less important.  If anything, the implication is the
reverse: Non-memory resources can be too important to leave to
the mercy of a garbage collector ignorant of their significance.

    The language has built-in support for managing memory but
lacks support for automatic management of database connections,
floating-license token reservations, font caches in the video
card, and on and on.  Perhaps Java's inventors felt that a general
framework for managing such an open-ended list of resources was
desirable (I said "perhaps," right?) but intractable.

    It could be argued that the destructor is just such a
framework.  I'm not well versed enough in language design to
make a convincing argument either way, and perhaps the argument
belongs on an advocacy group anyhow.  But for whatever reason,
Java does not have automatic destructors, and people who seize
on finalize() as a destructor substitute are doing themselves
no favors.

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Daniel Pitts - 08 Nov 2007 16:44 GMT
>> [...]
>> At the metalevel, there seems to be a curious disconnect here.  How are
[quoted text clipped - 23 lines]
> on finalize() as a destructor substitute are doing themselves
> no favors.

I think that in C++, the destructor is just a framework, although it too
might not be called if the process dies.  In Java, the framework is
try/finally, and that has the same potential problem.  However, it is
better than using finalize :-)

Signature

Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>

Joe Seigh - 09 Nov 2007 01:10 GMT
>> [...]
>> At the metalevel, there seems to be a curious disconnect here.  How are
[quoted text clipped - 15 lines]
> framework for managing such an open-ended list of resources was
> desirable (I said "perhaps," right?) but intractable.

They seem to be using it themselves.  See java.io.FileInputStream
docs and source.

>     It could be argued that the destructor is just such a
> framework.  I'm not well versed enough in language design to
[quoted text clipped - 3 lines]
> on finalize() as a destructor substitute are doing themselves
> no favors.

It's not a substitute.  I never said anything about using it instead
of explicit resource clean up.  It would be there for the exact same
reason java.io uses it.  

Signature

Joe Seigh

When you get lemons, you make lemonade.
When you get hardware, you make software.

Lew - 09 Nov 2007 03:37 GMT
> It's not a substitute.  I never said anything about using it instead
> of explicit resource clean up.  It would be there for the exact same
> reason java.io uses it.

Well, you seem hell-bent on using finalize() despite the best advice of
several people telling you not to.  You will suffer slower performance, higher
likelihood of memory issues and greater maintenance effort.

What is the "exact same reason java.io uses it"?  Please explain.

More importantly, what is the actual effect of the finalize() methods in those
library classes?

Is the net value provided a boon or a detriment?

Signature

Lew

Joe Seigh - 09 Nov 2007 11:30 GMT
>> It's not a substitute.  I never said anything about using it instead
>> of explicit resource clean up.  It would be there for the exact same
[quoted text clipped - 10 lines]
>
> Is the net value provided a boon or a detriment?

It's not best advice.  Your main objections seem to be that I'm thinking of
using it on everything which will slow things down, and that I'm going
to use it for deterministic destruction.  Neither is true.

As to why java.io, jdbc drivers, etc.. use it?  Maybe you've never had to
support a really large code base, or you have and enjoy getting paged at
3 am about something you can't really do anything about at the moment.
Which would you rather support?  An application that dies because some
combination of events caused it to execute a section of code that leaked
resources, or an application that could most likely recover those resources
before they ran out completely and issued diagnostic warnings instead.
Either you get this or you don't.

Signature

Joe Seigh

When you get lemons, you make lemonade.
When you get hardware, you make software.

Lew - 09 Nov 2007 13:41 GMT
> As to why java.io, jdbc drivers, etc.. use it?  Maybe you've never had to
> support a really large code base, or you have and enjoy getting paged at
[quoted text clipped - 4 lines]
> before they ran out completely and issued diagnostic warnings instead.
> Either you get this or you don't.

Sure, so using an idiom that increases the chances of bugs is your solution.
Go for it.

Go for it with the /ad hominem/ remarks, too.

Signature

Lew

Owen Jacobson - 09 Nov 2007 21:24 GMT
> >> It's not a substitute.  I never said anything about using it instead
> >> of explicit resource clean up.  It would be there for the exact same
[quoted text clipped - 23 lines]
> before they ran out completely and issued diagnostic warnings instead.
> Either you get this or you don't.

Do you trust the finalizer to release locks for you (or your caller)
if you forget to release them yourself?  And do you trust your
(caller's) code to be in a sane state if a forgotten lock is suddenly
released?

Do you trust the finalizer to close sockets for you (or your caller)
if you forget to release them yourself?  And do you trust your
(caller's) code to be in a sane state if a forgotten socket is
suddenly closed?

Do you trust the finalizer to close a file handle for you (or your
caller) if you forget to release them yourself?  And do you trust your
(caller's) code to be in a sane state if a forgotten file handle is
suddenly closed?

These are the sorts of questions that putting cleanup in the finalizer
should be bringing up for you.  In my own case, the answer to all of
them is "No.", so I don't put cleanup in the finalizer and do not rely
on finalizers to catch my own mistakes.  Finalizer "cleanup" only
changes one broken state (caller did not clean up properly) into
another broken state (caller did not clean up properly and may have
been unprepared for cleanup); it at best masks bugs and never fixes
them.

You can't truly enforce correctness on code that calls your code, and
attempting to is painful for both your code and your caller's code.
The garbage collector is particularly the wrong tool to try to use to
coerce others with, since it's tuned for managing exactly one
resource: memory.
Daniel Pitts - 08 Nov 2007 05:12 GMT
> The key word here is guarantee.  Obviously the proper way is to
> use a close method but the mere existence of such a method doesn't
[quoted text clipped - 3 lines]
> technical issues (beside the GC not offering any real time
> guarantees)?

Actually, its not an aesthetic at all.  The garbage collector treats
objects with finalize method differently, to the point where you can end
up with more garbage lying around. It can affect performance of the GC
to the point that you'll get OOME errors more frequently.  Also, if the
"this" reference escapes the finalize method, it can interfere with the
GC further.

Signature

Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>

Roger Lindsjö - 08 Nov 2007 15:35 GMT
> The key word here is guarantee.  Obviously the proper way is to
> use a close method but the mere existence of such a method doesn't
> guarantee its proper use.

But since finalize() is only guaranteed to run before trying to reuse
memory (not when object is made available for GC) you can't be certain
that it will run. It will also only run once, even if the object becomes
reachable after finalize() is invoked by the GC system.

Anecdote:
I have seen problems with this in one of my projects. Our installation
team complained that the server ran out of file handles, even if they
increased the ulimit. We asked what they did to trigger this and they
said "nothing, we just install the application, run our tests and then
suddenly we have exceptions and the application has aquired X file
handles. Sometimes it releases them again after a while, but then
exception comes back after a few more hours." X in this case was close
to the setting by ulimit. Initially we were unable to reproduce this but
we did find a class releasing a stream without closing it. The fix
remove problem.

So, why did they get the error while we never saw it during
development/testing? The class leaking the file handle was a monitoring
class that got to run every few seconds and hardly created any garbage
at all. So we used up file handles faster than memory IFF nothing else
ran. Of course that never happened for us, we would either run unit
tests or duration tests, but no test just started the application and
left it alone for >12 hours (the time it took us to trigger the error
once we knew the problem). Since the installation team often let this
application alone after installation they triggered the error. The
application was robust (apart from causing exceptions ;-) ) that after a
while it would run scheduled tasks that caused enough garbage to invoke
a GC which in turn caused the file handles to be released.

Now we actually include statistics of file handles during load and idle
and verify that we get expected results (which sometimes causes us to
change code or change our expectations).

//Roger Lindsjö
Daniel Pitts - 08 Nov 2007 16:46 GMT
>> The key word here is guarantee.  Obviously the proper way is to
>> use a close method but the mere existence of such a method doesn't
[quoted text clipped - 4 lines]
> that it will run. It will also only run once, even if the object becomes
> reachable after finalize() is invoked by the GC system.
Not only that, but the GC will put off collecting objects with
finalize() for much longer than normal.  It has adverse effects on the
whole algorithm.

Signature

Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>

Owen Jacobson - 08 Nov 2007 15:49 GMT
> >> I guess overriding finalize() isn't recommended too much because
> >> of it's adverse effect on GC performance.  But I assume that
[quoted text clipped - 26 lines]
> use a close method but the mere existence of such a method doesn't
> guarantee its proper use.

Neither does finalize().

The JVM is free to put off the collection of any given object for as
long as it wants, and to finalize objects in any order it feels like.
Objects that have not been collected when the program ends are likely
never to be finalized.  This is an intrinsic feature of Java's
finalization and can't be "worked around"; even the
runFinalizersAtExit method doesn't guarantee that finalizers will ever
be run (trivially, if the JVM dies of a KILL signal, it never gets a
chance to finalize anything, but there are other ways to cause it with
Sun's JVM).

Since Java has neither destructors nor C#'s using statement and
IDisposable interface, the idiom for using and discarding resources in
a controlled, guaranteed way is similar to this:

InputStream source = new FileInputStream ("foo.dat");
// do not put code here, ever.
try {
 readDataFrom (source);
} finally {
 source.close ();
}

Using finally like this does guarantee that, regardless of whether or
not the code represented by readDataFrom succeeds, the source stream
will be closed.  The same pattern works for any releaseable resources:
explicit locks, database connections, and so on.  It's a construct
that should be familiar to any Java programmer, and while it's a
little verbose, it works rather well and gives the programmer complete
control over the lifecycle of the resource.
Wojtek - 08 Nov 2007 15:51 GMT
Joe Seigh wrote :
> I guess overriding finalize() isn't recommended too much because
> of it's adverse effect on GC performance.  But I assume that
[quoted text clipped - 5 lines]
> it be avoided at all costs even if you leak file descriptors or
> whatnot?

I once tried closing a DB connection in the finalize().

What I found was that the DB connection was being closed while I was in
the middle of retrieving rows from the result set.

In other words, it appeared that the finalize method was being called
while the object was still being referenced.

I tested this by removing the close from finalize. This was the ONLY
change to the code. Oh yes, this behaviour also happened while stepping
through the code in the debugger.

close in finalize:
retrieve a random number of rows, but NEVER all the rows, usually about
25-40% of the rows

close in the "body" of the class and called explicitly:
100% every time.

I looked at the results of the tests, scratched my head, and decided
that I do not know enough about the GC process. And I vowed NEVER to
use finalize again :-)

I think this was Java version 1.4x with Apache 4.x

Signature

Wojtek :-)

Joe Seigh - 09 Nov 2007 01:01 GMT
> Joe Seigh wrote :
>> I guess overriding finalize() isn't recommended too much because
[quoted text clipped - 31 lines]
>
> I think this was Java version 1.4x with Apache 4.x

There was no problem with finalize.  What happened is the Statement and
ResultSet objects reference the connection internally.  You dropped the
reference to your enclosing object and it's finalizer closed the connection
while it was still being referenced and used by the ResultSet object.
You assumed that if your enclosing object wasn't being referenced, that
the objects contained in it were no longer being referenced.

Signature

Joe Seigh

When you get lemons, you make lemonade.
When you get hardware, you make software.

Wojtek - 09 Nov 2007 15:48 GMT
Joe Seigh wrote :

> There was no problem with finalize.  What happened is the Statement and
> ResultSet objects reference the connection internally.  You dropped the
> reference to your enclosing object and it's finalizer closed the connection
> while it was still being referenced and used by the ResultSet object.
> You assumed that if your enclosing object wasn't being referenced, that
> the objects contained in it were no longer being referenced.

In the following I left out all the try/catch/finally for brevity.

Note to new-comers. DO NOT use this pattern. It is flawed and you may
end up with many connections left open due to when the GC calls
finalize!

public class SQL
{
 private Connection conn;

 public SQL()
 {
   conn = Databasae.getConnection();
 }

 public Data getData()
 {
    Statement st;
    ResultSet rs;
    Data data = new Data();

    st = conn.createStatement();
    rs = st.executeQuery("select id,name from table");

    while ( rs.next() )
    {
       // code to fill data
    }

    return data;
 }

 public viod finalize()
 {
   conn.close();
 }
}

and to use this:

-------------------------------
SQL sql = new SQL();
Data data = sql.getData();
sql = null;
-------------------------------

So from my thinking:
- a connection is established in the constructor
- a method retrieves information
- when sql is set tu null, the only reference goes away.
- some time in the future the GC calls the finalize method.

In practice, the connection appeared to be closed before the "while (
rs.next() )" loop completed.

When I made the following change, the problem went away.

public class SQL
{
 public Data getData()
 {
    Connection conn;
    Statement st;
    ResultSet rs;
    Data data = new Data();

    conn = Databasae.getConnection();
    st = conn.createStatement();
    rs = st.executeQuery("select id,name from table");

    while ( rs.next() )
    {
       // code to fill data
    }

    conn.close();

    return data;
 }
}

I went back and forth several times between the to ways, and the
results were consistent.

Signature

Wojtek :-)

Roger Lindsjö - 09 Nov 2007 19:38 GMT
> Joe Seigh wrote :
>
[quoted text clipped - 18 lines]
>  {
>    conn = Databasae.getConnection();

I'm a bit qurious about this. What class is Databasae? Is it a pool, a
single connection (a pool of one ;-) or does it generate a new
connection for each request?

If it returns the same connection each time, then this scenario could
happen:

You create SQL(instance 1) which runs to completion. Later you create
SQL(instance 2) which gets a large ResultSet. Both instances now have a
reference to the same connection. Iteration over the ResultSet causes
some garbage so the GC decides to clean up memory. It finds SQL(instance
1) and runs finalize() on it which cases the connection to be closed.
Bad news for SQL(instance 2).

//Roger Lindsjö
Wojtek - 09 Nov 2007 20:34 GMT
Roger Lindsjö wrote :
>> Joe Seigh wrote :
>>
[quoted text clipped - 22 lines]
> connection (a pool of one ;-) or does it generate a new connection for each
> request?

It is a pool manager called proxool, Database is a wrapper class.

Signature

Wojtek :-)



Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.