Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / July 2009

Tip: Looking for answers? Try searching our database.

ArrayList.Iterator.remove()

Thread view: 
Donkey Hottie - 30 Jun 2009 21:50 GMT
Just noticed a 'hidden feature of Java' ;)

Removing items an ArrayList (at least with an Iterator) takes ages.

Better - while not obvious - solution is to create an new ArrayList with the
remaining elements.

So it seems. removal is futile.
Eric Sosman - 30 Jun 2009 22:07 GMT
> Just noticed a 'hidden feature of Java' ;)
>
> Removing items an ArrayList (at least with an Iterator) takes ages.

    If the ArrayList contains a lot of items, yes.  No, wait,
for "yes" read "of course" or even "obviously."

> Better - while not obvious - solution is to create an new ArrayList with
> the remaining elements.

    Depends on how many items there are, how many you're
deleting, and where they're positioned.

> So it seems. removal is futile.

    Nonsense.  No, wait, for "nonsense" read "balderdash" or
even "baloney."  If you brush your teeth with a broom and have
an unpleasant experience, it does not follow that brushing your
teeth is futile.

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Donkey Hottie - 30 Jun 2009 22:12 GMT
>> Just noticed a 'hidden feature of Java' ;)
>>
[quoted text clipped - 16 lines]
> and have an unpleasant experience, it does not follow
> that brushing your teeth is futile.

* Loaded 223673 addresses
* Sorting...
* Merging...
* Merged 22577 address ranges.

Removed 22577 items from 223673. Don't know it that is many or not.
Lew - 30 Jun 2009 22:40 GMT
> * Loaded 223673 addresses
> * Sorting...
> * Merging...
> * Merged 22577 address ranges.
>
> Removed 22577 items from 223673. Don't know it that is many or not.

I would expect, based on the extremely non-hidden documentation of
ArrayList, for that to take time roughly

T = (22577 * 223673 / 2) * k

where 'k' is the time to copy an element from one array location to
another.

If 'k' is around 100 nanoseconds, T would be around 252 seconds, or
somewhat over 4 minutes.  A 'k' of 10 ns would require about 25
seconds of removal time.

--
Lew
Donkey Hottie - 30 Jun 2009 22:54 GMT
>> * Loaded 223673 addresses
>> * Sorting...
[quoted text clipped - 15 lines]
> seconds, or somewhat over 4 minutes.  A 'k' of 10 ns
> would require about 25 seconds of removal time.

Great ;)

I'm writing a rival to a Windows application, which does that merge at least
5 minutes in a Athlon XP 1900+ machine. My Java implementation does it now
in ten seconds in a Pentium Pro 3 machine.

And I though as a C++ programmer (for 10 years) that Java is sluggish..
Eric Sosman - 30 Jun 2009 23:32 GMT
>>> * Loaded 223673 addresses
>>> * Sorting...
[quoted text clipped - 23 lines]
>
> And I though as a C++ programmer (for 10 years) that Java is sluggish..

    As almost always, you get a lot more improvement out of
choosing the right data structures and algorithms than out of
shaving cycles or even out of choosing implementation language.

    You haven't fully explained what you're trying to do, but
at a guess you're collecting a big pile of numeric "addresses"
from somewhere, sorting them, possibly eliminating duplicates,
and finally combining groups of neighboring addresses into
"ranges."  ArrayList doesn't strike me as a good candidate for
the elimination and combining stages, precisely because of the
time required to squeeze out the vacated slots.  If you *must*
use ArrayList (for reasons the sketchy problem description does
not reveal), consider working on it from the end toward the
beginning instead of from the beginning toward the end.  (But
since you're removing only 10% of the total, I don't hold out
a lot of hope for a huge improvement therefrom.)

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Roedy Green - 01 Jul 2009 00:02 GMT
On Tue, 30 Jun 2009 18:32:21 -0400, Eric Sosman
<esosman@ieee-dot-org.invalid> wrote, quoted or indirectly quoted
someone who said :

> ArrayList doesn't strike me as a good candidate for
>the elimination and combining stages, precisely because of the
>time required to squeeze out the vacated slots.

It is very quick if you create a new array.  See
http://mindprod.com/products1.html#SORTED for a set of classes for
merging and deduping etc.  Unfortunately, the classes were written
before generics were introduced.
Signature

Roedy Green Canadian Mind Products
http://mindprod.com

"Deer hunting would be fine sport, if only the deer had guns."
~ William S. Gilbert of Gilbert and Sullivan

Eric Sosman - 01 Jul 2009 02:05 GMT
> On Tue, 30 Jun 2009 18:32:21 -0400, Eric Sosman
> <esosman@ieee-dot-org.invalid> wrote, quoted or indirectly quoted
[quoted text clipped - 5 lines]
>
> It is very quick if you create a new array.  [...]

    Assuming the items being removed are fairly numerous.  (As
it seems they are, although the O.P. let slip this information
only in follow-ups and not in the original "removal is futile"
post.)

    If only a few items are being deleted, especially if they
are known to appear near the end, the work involved in sliding
all their successors one place to the left may well be less than
the work of copying "all except" to an entirely new array.  Horses
for courses, and don't blame the horse if he falters when you
ride him up the ski jump.

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Arne Vajhøj - 30 Jun 2009 23:59 GMT
>>> * Loaded 223673 addresses
>>> * Sorting...
[quoted text clipped - 23 lines]
>
> And I though as a C++ programmer (for 10 years) that Java is sluggish..

I would expect a C++ program using STL vector to have the same
behavior.

Arne
Joshua Cranmer - 01 Jul 2009 01:55 GMT
> I would expect a C++ program using STL vector to have the same
> behavior.

He should have used the STL list instead ;-).

Signature

Beware of bugs in the above code; I have only proved it correct, not
tried it. -- Donald E. Knuth

Arne Vajhøj - 01 Jul 2009 02:36 GMT
>> I would expect a C++ program using STL vector to have the same
>> behavior.
>
> He should have used the STL list instead ;-).

Yep.

But then LinkedList is the solution in Java.

Arne
Arne Vajhøj - 30 Jun 2009 23:57 GMT
>>> Just noticed a 'hidden feature of Java' ;)
>>>
[quoted text clipped - 23 lines]
>
> Removed 22577 items from 223673. Don't know it that is many or not.

It is many.

If all the items were removed first in the list you would have moved
approx. 4.8 billion elements because ArrayList is backed by an array
and need to move things around after a delete.

Arne
Patricia Shanahan - 30 Jun 2009 22:16 GMT
>> Just noticed a 'hidden feature of Java' ;)
>>
[quoted text clipped - 15 lines]
> an unpleasant experience, it does not follow that brushing your
> teeth is futile.

Also, if access to a List is dominated by iterators with remove,
LinkedList is likely to be more efficient than ArrayList. ArrayList is
optimized for indexed access.

Patricia
Donkey Hottie - 30 Jun 2009 22:31 GMT
> Also, if access to a List is dominated by iterators with
> remove, LinkedList is likely to be more efficient than
> ArrayList. ArrayList is optimized for indexed access.

Thanks!

I was not aware of that kind of a List being in Java library (thought it
would be an Apache solution or something).

LinkedList now belongs to my toolkit allright.
Lew - 01 Jul 2009 01:05 GMT
>> Also, if access to a List is dominated by iterators with
>> remove, LinkedList is likely to be more efficient than
[quoted text clipped - 4 lines]
> I was not aware of that kind of a List being in Java library (thought it
> would be an Apache solution or something).

This is where being a Javadoc junkie pays off.

<http://java.sun.com/javase/6/docs/api/java/util/package-summary.html>

There really is no reason not to be aware of the basic collection classes.

Signature

Lew

Roedy Green - 01 Jul 2009 06:58 GMT
>There really is no reason not to be aware of the basic collection classes.

For a quick overview of what each Collection is good for, see
http://mindprod.com/jgloss/collection.html
Signature

Roedy Green Canadian Mind Products
http://mindprod.com

"Deer hunting would be fine sport, if only the deer had guns."
~ William S. Gilbert of Gilbert and Sullivan

Sabine Dinis Blochberger - 03 Jul 2009 12:01 GMT
> >> Also, if access to a List is dominated by iterators with
> >> remove, LinkedList is likely to be more efficient than
[quoted text clipped - 10 lines]
>
> There really is no reason not to be aware of the basic collection classes.

I actually like to also refer to Suns Java Tutorial, because it explains
things in "plain english". Sometimes the JavaDoc doesn't give much clue
on how or when to use a certain class, although often it will link to
the tutorial.

<http://java.sun.com/docs/books/tutorial/>

And to collections
<http://java.sun.com/docs/books/tutorial/collections/implementations/index.html>

Signature

Op3racional - www.op3racional.eu
---------------------
If you're reading this, you're on Usenet
<http://oakroadsystems.com/genl/unice.htm>

markspace - 01 Jul 2009 01:11 GMT
>> ArrayList. ArrayList is optimized for indexed access.
>
> Thanks!
>
> I was not aware of that kind of a List being in Java library (thought it

What?!  Are you kidding me?  I'm sorry but this is bull****.  How the
hell can you program in Java at all with out at least bumping into the
Collections classes?  This is really amazing to me.  They're in every
tutorial.  It's basic algorithms and should be instantly familiar to
anyone who made it through their lower division course work.... I'm just
aghast.  Seriously.

Especially a C++ programmer should be on the lookout for something in a
new language to replace the STL, and Collections is a big part of Java's
answer to the STL.
Donkey Hottie - 01 Jul 2009 11:40 GMT
>>> ArrayList. ArrayList is optimized for indexed access.
>>
[quoted text clipped - 13 lines]
> something in a new language to replace the STL, and
> Collections is a big part of Java's answer to the STL.

I was aware of the Collections, with ArrayList, but not *LinkedList*. It's a
late addition, and I have moved out from Java 1.4.2 only lately... Now using
Java 6.
Eric Sosman - 01 Jul 2009 13:15 GMT
> I was aware of the Collections, with ArrayList, but not *LinkedList*.
> It's a late addition, and I have moved out from Java 1.4.2 only
> lately... Now using Java 6.

    java.util.LinkedList is "since 1.2" in the Javadoc.

Signature

Eric Sosman
esosman@ieee-dot-org.invalid

Lew - 01 Jul 2009 13:24 GMT
> I was aware of the Collections, with ArrayList, but not *LinkedList*.
> It's a late addition, and I have moved out from Java 1.4.2 only
> lately... Now using Java 6.

Yeah, it wasn't introduced until Java 1.2, ten and a half years ago.

Signature

Lew

Patricia Shanahan - 01 Jul 2009 15:39 GMT
>> I was aware of the Collections, with ArrayList, but not *LinkedList*.
>> It's a late addition, and I have moved out from Java 1.4.2 only
>> lately... Now using Java 6.
>
> Yeah, it wasn't introduced until Java 1.2, ten and a half years ago.

This sort of issue is one of the reasons why I read this newsgroup
regularly. It is especially valuable for a solo programmer. I know a lot
of things exist because people have asked questions about them, or they
have been recommended as solutions.

Patricia
Lew - 01 Jul 2009 17:10 GMT
Donkey Hottie wrote:
>>> I was aware of the Collections, with ArrayList, but not *LinkedList*.
>>> It's a late addition, and I have moved out from Java 1.4.2 only
>>> lately... Now using Java 6.

Lew wrote:
>> Yeah, it wasn't introduced until Java 1.2, ten and a half years ago.

> This sort of issue is one of the reasons why I read this newsgroup
> regularly. It is especially valuable for a solo programmer. I know a lot
> of things exist because people have asked questions about them, or they
> have been recommended as solutions.

You make a very good point.  However, the collections classes are
among the Java features that are quite fundamental, and when working
with such things as collections one should be in the habit of
reviewing the Javadocs to see what exists already.  For one accustomed
to using the Javadocs it would be quite clear after having been
alerted to the existence of a decade-old API that the class is not
new, even if they didn't discover the class until having been alerted
to it.

Usenet is useful, but not the best as the primary source of Java
knowledge, let alone the only one.  To achieve competence in Java
programming, let alone virtuosity, one must be in the habit of reading
the Javadocs, the tutorials, search engine results, and yes, even
books.  It helps to be aware of third-party library sources, too, like
Apache Commons (which has even more collections classes), sourceforge
and the like.

LinkedList, HashMap, java.util.Collections and other such have been
officially part of the Java API since 1998.  Like, say, large parts of
the java.io and javax.swing packages, there's precious little reason
for a professional Java programmer not to be at least aware of their
existence, nor to think that they're brand new.

--
Lew
Knute Johnson - 01 Jul 2009 23:19 GMT
> Donkey Hottie wrote:
>>>> I was aware of the Collections, with ArrayList, but not *LinkedList*.
[quoted text clipped - 34 lines]
> --
> Lew

I'm with Patricia on this.  I work by myself and while I read books, the
docs and the tutorials, the API is staggering in its size and
complexity.  The news groups have been invaluable for me in my work.  On
the Collections issue specifically, I've used them for years but I don't
know all the ins and outs of what is more efficient for a particular
type of operation.  This thread gave me a lot of valuable information
that I will use in the future and now to look at code already written
with an eye to better performance.

I am not however making any excuses for the OPs ignorance of the API.
You are right about that, he should have some clue.  We do get folks in
here though that have never written a lick of Java code and need to fix
something.  I have to fix Perl once in a while (and I need the three
books I have for that) and have to learn new languages periodically.
Sometimes you can be pretty clueless and a good pointer or two will get
you going.

The two main reasons I inhabit these newsgroups, is that I received so
much help here when I really needed it and that I learn so many new
things by reading the posts.  Not as many today as in the beginning but
like this post some valuable tidbits here and there.

Signature

Knute Johnson
email s/nospam/knute2009/

Lew - 30 Jun 2009 22:10 GMT
> Just noticed a 'hidden feature of Java' ;)
>
[quoted text clipped - 4 lines]
>
> So it seems. removal is futile.

Not so hidden, really.

<http://java.sun.com/javase/6/docs/api/java/util/ArrayList.html>
> The size, isEmpty, get, set, iterator, and listIterator operations run
> in constant time. The add operation runs in amortized constant time, that
> is, adding n elements requires O(n) time. All of the other operations run
> in linear time (roughly speaking).

That's for *each* removal.

That's because 'ArrayList#remove()' (through an iterator or
otherwise)
> [s]hifts any subsequent elements to the left (subtracts one from their indices).
<http://java.sun.com/javase/6/docs/api/java/util/ArrayList.html#remove
(int)>

The documentation for the collection classes usually indicates the big-
O time of the common operations.

I would expect copying to take longer, however, depending on where one
is in the iteration.  Perhaps there's extra machinery in the iterator
logic to normalize the iterator's position within the list.

Or are you copying individual elements as you iterate, skipping over
the "removed" ones?  That should be much faster than multiple
removals.

--
Lew
Donkey Hottie - 30 Jun 2009 22:23 GMT
.

> I would expect copying to take longer, however, depending
> on where one
[quoted text clipped - 8 lines]
> multiple
> removals.

Well, after I discovered the slow speed of removal, I rewrote the method now
creates a new List and always just adds the suitable elements and replaces
the List instance.

10% of the original List seem to be removed, and it is 1000 times faster
now.
Knute Johnson - 30 Jun 2009 23:13 GMT
> .
>>
[quoted text clipped - 17 lines]
> 10% of the original List seem to be removed, and it is 1000 times faster
> now.

I'd be really curious to know if using the LinkedList is significantly
faster than ArrayList.  If you try it, please post back.

Thanks,

Signature

Knute Johnson
email s/nospam/knute2009/

Arne Vajhøj - 30 Jun 2009 23:57 GMT
>> I would expect copying to take longer, however, depending
>> on where one
[quoted text clipped - 15 lines]
> 10% of the original List seem to be removed, and it is 1000 times faster
> now.

Most likely it would be better with another data type than
ArrayList.

Arne
Kevin McMurtrie - 01 Jul 2009 06:03 GMT
> Just noticed a 'hidden feature of Java' ;)
>
[quoted text clipped - 4 lines]
>
> So it seems. removal is futile.

Skipped school?  This is in the basics of information science.  It has
nothing to do with Java.

When I think of hidden Java features I think of secret accessor methods,
generics insanity, -XX switches, FinalReference, and in-place math
operators lacking narrowing checks.

Signature

I will not see your reply if you use Google.

Lew - 01 Jul 2009 13:26 GMT
> generics insanity,

To what insanity do you refer?

Signature

Lew

charlesbos73 - 01 Jul 2009 16:53 GMT
> > generics insanity,
>
> To what insanity do you refer?

erasure?  (say as opposed to the way they're done in C#),
making them useless from an OO point of view?

public interface UselessGenerics extends Handable<A>, Handable<B> {

"Duplicate class: '.....Handable'

AAAAAAlllllrrrriiiigggghhhhhttttt.

It's just like the broken Java equals() concept: it "works" only
as long as you're doing procedural programming.

As soon as you want to some real OOA/OOD->OOP translation you're
out of luck.

But I'll give it to you: for 99.9% of all the Java programs out
there are just glorified procedural programs it's no big deal.
Mayeul - 01 Jul 2009 17:16 GMT
>>> generics insanity,
>> To what insanity do you refer?
>
> erasure?  (say as opposed to the way they're done in C#),

Point taken I guess. No erasure would have horribly broken Java 1.4 code
and previous, so it is necessary. Still, it is a flaw.

> making them useless from an OO point of view?

Seems like an overstatement. The commodity of generics fairly
facilitates OO design, which is just as valid as an 'OO point of view'.

> public interface UselessGenerics extends Handable<A>, Handable<B> {
>
> "Duplicate class: '.....Handable'
>
> AAAAAAlllllrrrriiiigggghhhhhttttt.

Just because this doesn't work, doesn't make everything useless.

> It's just like the broken Java equals() concept: it "works" only
> as long as you're doing procedural programming.

It's not the first time you criticized Java Object.equals(), and you've
been asked for clarifications, but I didn't see them. I still fail to
understand how not OO or how anti-OO that is.

> As soon as you want to some real OOA/OOD->OOP translation you're
> out of luck.

[Citation needed]

> But I'll give it to you: for 99.9% of all the Java programs out
> there are just glorified procedural programs it's no big deal.

Actually that might be true, considering that the vast majority of
programming done on the whole world is not done by skilled software
designers.

Same for just about any well-known language, I guess.

--
Mayeul
Lew - 01 Jul 2009 18:48 GMT
charlesbos73 wrote:
>> erasure?  (say as opposed to the way they're done in C#),

> Point taken I guess. No erasure would have horribly broken Java 1.4 code
> and previous, so it is necessary. Still, it is a flaw.

While there are definitely times when having generics information at
run time would help, the fact that it's compile-time only does benefit
in that it forces one to handle type issues at compile time, when bugs
are much cheaper to fix than at run time.  In practice, the need for
run-time generification is only for corner cases; most of the time
it's completely sufficient to have only compile-time checking.

People misunderstand the generics mechanism.  Generics support type
analysis, and type analysis can be quite tricky.  Hence, use of
generics can be tricky.  However, once you get all the generics
compiler messages ironed out, you have a self-documented ironclad type
contract in your code.  It's well worth the effort.

charlesbos73 wrote:
>> It's just like the broken Java equals() concept: it "works" only
>> as long as you're doing procedural programming.

> It's not the first time you criticized Java Object.equals(), and you've

And probably not the last.

> been asked for clarifications, but I didn't see them. I still fail to
> understand how not OO or how anti-OO that is.

He's just being trollish.  Making inflammatory comments without
evidence or logic is one of those classic trollish Usenet behaviors.

Others have responded to his criticism that Java is not object-
oriented, which, of course, it is, and "charlesbos73" was equally
silent regarding their responses.  At any rate, I haven't seen anyone
respond to any of his responses.  For example, his misrepresentation
of statements from /Effective Java/ was called to account.

Polymorphism is one of Grady Booch's six pillars of object orientation
- by that standard 'Object.equals()' definitely fits the O-O paradigm.

I see no way in which Java violates object orientation except perhaps
the support for static members.  Java is clearly an object-oriented
language.  (OK, Smalltalk fans, settle down.)

--
Lew
Eric Sosman - 01 Jul 2009 19:31 GMT
> charlesbos73 wrote:
>>> erasure?  (say as opposed to the way they're done in C#),
[quoted text clipped - 6 lines]
> in that it forces one to handle type issues at compile time, when bugs
> are much cheaper to fix than at run time.

    Yes, compile-time checking is Good.  But it doesn't follow that
run-time checking in addition to compile-time checking would be Bad.

> In practice, the need for
> run-time generification is only for corner cases; most of the time
> it's completely sufficient to have only compile-time checking.

    Some of the corners are fairly gently rounded, as witness the
number of questions that arise here about them.  Perhaps the two
most common "smooth corners" are constructing instances of and
making arrays of the parameterized type:

    class Generic<T> {
       T instance = new T();   // no good
       T[] array = new T[42];  // no good
    }

> People misunderstand the generics mechanism.  Generics support type
> analysis, and type analysis can be quite tricky.  Hence, use of
> generics can be tricky.

    A-men, brother!  I recall that when generics first came along
I decided I'd retrofit them into some of my existing code that used
Object references and a lot of casts.  NetBeans 3.mumble kept drawing
squiggly red lines under usages I thought were valid, and I simply
couldn't comprehend what was wrong.  I tried a javac compilation from
the command line in the faint hope of getting a more comprehensible
diagnostic, and lo! javac accepted my code happily!

    NetBeans 5.mumble continued to protest my acceptable-to-javac
code, and it wasn't until NB 6 came along that the squiggly lines
finally went away.  If even the tools developers can't get generics
right, what hope is there for mere mortals?

> I see no way in which Java violates object orientation except perhaps
> the support for static members.  [...]

    Some have disparaged its use of non-objectified primitive types.
Zealotry is its own reward, I guess.

Signature

Eric.Sosman@sun.com

Arved Sandstrom - 01 Jul 2009 20:41 GMT
[ SNIP ]

>     Some of the corners are fairly gently rounded, as witness the
> number of questions that arise here about them.  Perhaps the two
[quoted text clipped - 5 lines]
>         T[] array = new T[42];  // no good
>     }
[ SNIP ]

About the closest you can get is to use
java.lang.reflect.Array.newInstance(Class<?>, int) or
java.lang.reflect.Array.newInstance(Class<?>, int[]). It's not perfect,
as pointed out by
Goetz(http://www.ibm.com/developerworks/java/library/j-jtp01255.html,
about halfway down the page).

AHS
Kevin McMurtrie - 02 Jul 2009 07:59 GMT
> > generics insanity,
>
> To what insanity do you refer?

Simple generics work fine but all hell breaks loose on complex data. The
solution just isn't very robust. This sums it up well:
http://www.ibm.com/developerworks/java/library/j-jtp01255.html

Don't even get me started on ReferenceQueue generics.

It also forgets to mention that each genericized method has a
non-genericized method automatically created by the source compiler:

class Foo extends HashSet<Integer>{}
class Bar<T> extends HashMap<Integer, T>{}
class Moo<T> extends HashMap<Integer, T>
{
  @Override public T put(Integer key, T value)
  {
     return super.put(key, value);
  }
}

new Foo().add("hello");             //Compile Error
new Bar<String>().put("hello", "there"); //Compile Error
new Bar().put("hello", "there");   //OK... until later.

//Drumroll....
new Moo().put("hello", "there");
//ClassCastException on a bogus line number

Signature

I will not see your reply if you use Google.

Lew - 02 Jul 2009 12:55 GMT
>>> generics insanity,

Lew wrote:
>> To what insanity do you refer?

> Simple generics work fine but all hell breaks loose on complex data. The
> solution just isn't very robust. This sums it up well:
> http://www.ibm.com/developerworks/java/library/j-jtp01255.html

I don't see insanity here.

> Don't even get me started on ReferenceQueue generics.
>
> It also forgets to mention that each genericized method has a
> non-genericized method automatically created by the source compiler:

The output of the compiler is generics-free, of course.  This is well documented.

> class Foo extends HashSet<Integer>{}
> class Bar<T> extends HashMap<Integer, T>{}
[quoted text clipped - 7 lines]
>
> new Foo().add("hello");             //Compile Error

Duhhh.  That's the purpose of generics.

> new Bar<String>().put("hello", "there"); //Compile Error

Duhhh.  That's the purpose of generics.

> new Bar().put("hello", "there");   //OK... until later.
>
> //Drumroll....
> new Moo().put("hello", "there");
> //ClassCastException on a bogus line number

Well, duhhh.  You used a raw type, which does create a compiler warning.  One
is thoroughly advised not to mix raw types and generics.  That
ClassCastException is exactly what one would have gotten before generics were
added to Java.

What you illustrate here is not generics insanity but a typical programmer error.

Signature

Lew

Kevin McMurtrie - 02 Jul 2009 16:07 GMT
> >>> generics insanity,
>
[quoted text clipped - 46 lines]
> What you illustrate here is not generics insanity but a typical programmer
> error.

The last two have been problems when updating existing code in a large
project.  Regardless of whether classes Moo or Bar are constructed with
generics, the Map key is clearly defined as Integer.  The source
compiler added a hidden method that allows compilation of code that can
not run.

Compiler warnings are great for new code, but not so great for the
millions of lines of old code.

Signature

I will not see your reply if you use Google.

Peter Duniho - 02 Jul 2009 17:43 GMT
> [...]
> Compiler warnings are great for new code, but not so great for the
> millions of lines of old code.

I'd say they're pretty good for old code, even millions of lines, if you  
actually go _fix_ them.

Sheer quantity of code isn't really a justification for leaving errors in  
it.
Kevin McMurtrie - 03 Jul 2009 06:37 GMT
> > [...]
> > Compiler warnings are great for new code, but not so great for the
[quoted text clipped - 5 lines]
> Sheer quantity of code isn't really a justification for leaving errors in  
> it.

Tell your boss you want to spend a full year fixing Java 1.5 warnings
because there's a 1 in 600 chance that each is a bug.  Next convince
your boss that the risk in fixing them is lower than the original 1 in
600 odds.

Signature

I will not see your reply if you use Google.

Lew - 03 Jul 2009 06:49 GMT
Peter Duniho wrote:
>> Sheer quantity of code isn't really a justification for leaving errors in  
>> it.

> Tell your boss you want to spend a full year fixing Java 1.5 warnings
> because there's a 1 in 600 chance that each is a bug.  Next convince
> your boss that the risk in fixing them is lower than the original 1 in
> 600 odds.

Oh, yeah, leave the bugs in the code.  That's a *real* professional attitude!

Signature

Lew

Patricia Shanahan - 03 Jul 2009 14:08 GMT
> Peter Duniho wrote:
>>> Sheer quantity of code isn't really a justification for leaving
[quoted text clipped - 7 lines]
> Oh, yeah, leave the bugs in the code.  That's a *real* professional
> attitude!

I agree that it is a really professional attitude. *Every* change to
code carries a risk of creating a bug. Clean existing code and a good
development process reduce the risk, but it cannot be reduced to zero.
Every change also has a cost in programmer time and effort that could
have been spent on something else.

I've seen important releases, with real bug fixes and features the users
wanted, delayed because someone decided to clean up a harmless departure
from technical desirability, and introduced a new bug in the process.

A professional programmer should step beyond purely technical issues to
think about whether the risk and cost associated with a change are
justified by the benefit.

I do not mean to imply one should never clean up warnings etc. I
strongly prefer warning-free code. Neither the cost nor the benefit of
clean-up is zero, so it is a non-trivial decision.

Patricia
Arved Sandstrom - 03 Jul 2009 15:08 GMT
>> Peter Duniho wrote:
>>>> Sheer quantity of code isn't really a justification for leaving
[quoted text clipped - 27 lines]
>
> Patricia

One thing I learned early in my career, which is related to this
subject, is, as much as it pains you to see spelling errors in method
names and variable names (for example, coders using "principal" when
they meant "principle", and vice versa, or "similiar" instead of the
correct "similar"), don't take advantage of your IDE's refactoring
capability to correct everything. :-) It's amazing how badly this can
bite you.

In the larger sense I agree with Patricia. Once a piece of code is
baselined, you can see an obvious error staring you right in the face
while working on something else, but you have to resist the temptation
to fix it. Code somewhere else may rely on that error. You need to have
permission to make the fix, after explaining why you should fix it.

AHS
Peter Duniho - 03 Jul 2009 16:55 GMT
> [...]
> In the larger sense I agree with Patricia. Once a piece of code is  
> baselined, you can see an obvious error staring you right in the face  
> while working on something else, but you have to resist the temptation  
> to fix it. Code somewhere else may rely on that error. You need to have  
> permission to make the fix, after explaining why you should fix it.

The points you and Patricia both make are absolutely correct.  One should  
never approach changes to code lightly.

But, that doesn't mean there's no value in deciding to spend the time to  
tackle warnings in the code.

(Of course, this entire discussion assumes a code base that somehow  
managed to get to "millions of lines of old code" without it _ever_  
occurring to someone that they ought to be compiling with  
warnings-as-errors, or otherwise requiring the code be warning-free.  I  
suppose in that environment, it might actually be a better move for a  
competent programmer to just find a job at a place that doesn't suck so  
much).

Pete
Lew - 03 Jul 2009 17:14 GMT
>> Peter Duniho wrote:
>>>> Sheer quantity of code isn't really a justification for leaving
[quoted text clipped - 25 lines]
> strongly prefer warning-free code. Neither the cost nor the benefit of
> clean-up is zero, so it is a non-trivial decision.

What you suggest is valid and certainly not using "sheer quantity of code" as
a justification.  However, the attitude that "mere" warnings are acceptable
does lead to real bugs that cause trouble in production.  Where did Kevin's "1
in 600 chance" assessment come from?  Statistical analysis?  I feel certain
not, rather, it was pulled from a particular posterior region.

99% (another posteriorily postulated probability) of the time someone
recommends simply not fixing warnings they are ignoring real bugs.

I've seen this in my current large-team project and in past such projects,
where a request for customer A's data resulted in data from customer B's
records because people ignored "trivial" warnings, or merely left out simple
code analysis that would have revealed trouble that didn't raise a compiler
warning.

A professional programmer should step beyond purely arbitrary (and specious)
characterizations of risk and cost associated with a change to understand that
*every* failure to change code that has warnings carries a risk of
perpetuating a bug.  I agree that risk cannot be reduced to zero, but leaving
known warnings in code increases that risk.

Code should not be released into the code base in the first place with
warnings showing.  If that's done, then the question of the risk of change
becomes moot, because there's nothing to change.  When that principle is
ignored, the technical debt mounts until people can aver that the "risk of
cleaning up" is too high to shoulder.  In practice, I've seen much more
failure to perform correctly arise from leaving known bugs in than from
cleaning them up.

Once they're in, then the cost of repair is high, but there are mitigation
strategies even so.  Again using my current project as an example, where we
have multiple warnings (from the compiler and from Findbugs, a /sine qua non/
for robust code) the reviewers recommend to management a strategy that cleans
up the most critical bugs right away, but defers others to a refactoring
timetable that also improves fundamental algorithms or updates to later
features (e.g., JPA over old-style ORM code).  Combination strategies like
fixing the algorithm often reap much larger benefits than simple patches for
similar cost, including simplification of the code to reduce the risk of
introducing new bugs.

Just because one maintainer was careless with a fix doesn't mean that the fix
itself was a bad idea.  Zero defects should be the goal rather than
resignation to poor quality.  You can't use "it's impossible" as an excuse not
to aim for it.  And you most certainly can't use arbitrary and unsupported
fake statistics to support a cost/benefit analysis.

Signature

Lew

Peter Duniho - 03 Jul 2009 16:51 GMT
> [...]
>> Sheer quantity of code isn't really a justification for leaving errors  
[quoted text clipped - 3 lines]
> Tell your boss you want to spend a full year fixing Java 1.5 warnings
> because there's a 1 in 600 chance that each is a bug.

Been there, done that.  We fixed the warnings.  If it takes a full year,  
then your boss may want to put more than one guy on the project.

> Next convince
> your boss that the risk in fixing them is lower than the original 1 in
> 600 odds.

I didn't waste any time making up numbers like "1 in 600 chance" or "lower  
than the original".  Complete non-issue.

Pete
Lew - 02 Jul 2009 22:30 GMT
>>> class Moo<T> extends HashMap<Integer, T>
>>> {
[quoted text clipped - 3 lines]
>>>    }
>>> }

...
>>> new Moo().put("hello", "there");
>>> //ClassCastException on a bogus line number
...
> Regardless of whether classes Moo or Bar are constructed with
> generics, the Map key is clearly defined as Integer.  The source
> compiler added a hidden method that allows compilation of code that can
> not run.

It's not a "hidden" method, it's
'public Object put( Object key, Object value )',
as is always the case when you erase generics, and is heavily documented as such.

The 'Map' key is based on extending 'HashMap<Integer, T>', which erases to
'HashMap <Object, Object>', and is therefore not "clearly defined as Integer"
at run time.  You call this "insanity", but it's really just plain old simple
erasure.  This is the common programmer error to which I alluded upthread, and
the inevitable consequence of mixing raw and generics types.  You insert a
'String' key into the map, then try to extract it to an 'Integer' target, thus
yielding the 'ClassCastException', just as documented and expected for this
scenario.  Again, this is what would happen if you tried this prior to Java 5.

It makes no sense to break the rules, get the exact result that is documented
for breaking the rules, then complain that you get that result.

Signature

Lew

Tom Anderson - 01 Jul 2009 23:16 GMT
>> Just noticed a 'hidden feature of Java' ;)
>
> When I think of hidden Java features I think of secret accessor methods,
> generics insanity, -XX switches, FinalReference, and in-place math
> operators lacking narrowing checks.

FinalReference is new to me. Now i know what it is, but that's it - what
can you do with it?

tom

Signature

How's it going to end?

Kevin McMurtrie - 02 Jul 2009 07:42 GMT
> >> Just noticed a 'hidden feature of Java' ;)
> >
[quoted text clipped - 6 lines]
>
> tom

Try to avoid them!  When you override finalize(), each instantiation of
your object creates a Finalizer that's a subclass of FinalReference.  A
Thread polls its ReferenceQueue and calls to finalize() during the GC
process.  Overriding finalize(), even with an empty method, on
frequently created objects can be a catastrophic drain on performance
and memory.

Signature

I will not see your reply if you use Google.

Lew - 02 Jul 2009 13:00 GMT
>>>> Just noticed a 'hidden feature of Java' ;)
>>> When I think of hidden Java features I think of secret accessor methods,
[quoted text clipped - 9 lines]
> frequently created objects can be a catastrophic drain on performance
> and memory.

Could someone please point to a FinalReference reference?

Signature

Lew

Arved Sandstrom - 02 Jul 2009 14:19 GMT
>>>>> Just noticed a 'hidden feature of Java' ;)
>>>> When I think of hidden Java features I think of secret accessor
[quoted text clipped - 11 lines]
>
> Could someone please point to a FinalReference reference?

I couldn't find one myself. Looking at the code for
java.lang.ref.Finalizer, which extends java.lang.ref.FinalReference, is
probably the best way to figure it out.

AHS
Lew - 02 Jul 2009 22:32 GMT
Lew wrote:
>> Could someone please point to a FinalReference reference?

> I couldn't find one myself. Looking at the code for
> java.lang.ref.Finalizer, which extends java.lang.ref.FinalReference, is
> probably the best way to figure it out.

So, once again, not a hidden *feature* but a hidden *implementation detail*.

I thought one of the tenets of O-O programming was to hide implementation details.

Signature

Lew

Arved Sandstrom - 03 Jul 2009 00:34 GMT
> Lew wrote:
>>> Could someone please point to a FinalReference reference?
[quoted text clipped - 8 lines]
> I thought one of the tenets of O-O programming was to hide
> implementation details.

Both Finalizer and FinalReference are package private in java.lang.ref,
so pretty clearly these are not classes that the masses are meant to
use. In other words, definitely implementation details.

AHS


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2010 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.