Java Forum / General / July 2009
ArrayList.Iterator.remove()
Donkey Hottie - 30 Jun 2009 21:50 GMT Just noticed a 'hidden feature of Java' ;)
Removing items an ArrayList (at least with an Iterator) takes ages.
Better - while not obvious - solution is to create an new ArrayList with the remaining elements.
So it seems. removal is futile.
Eric Sosman - 30 Jun 2009 22:07 GMT > Just noticed a 'hidden feature of Java' ;) > > Removing items an ArrayList (at least with an Iterator) takes ages. If the ArrayList contains a lot of items, yes. No, wait, for "yes" read "of course" or even "obviously."
> Better - while not obvious - solution is to create an new ArrayList with > the remaining elements. Depends on how many items there are, how many you're deleting, and where they're positioned.
> So it seems. removal is futile. Nonsense. No, wait, for "nonsense" read "balderdash" or even "baloney." If you brush your teeth with a broom and have an unpleasant experience, it does not follow that brushing your teeth is futile.
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Donkey Hottie - 30 Jun 2009 22:12 GMT >> Just noticed a 'hidden feature of Java' ;) >> [quoted text clipped - 16 lines] > and have an unpleasant experience, it does not follow > that brushing your teeth is futile. * Loaded 223673 addresses * Sorting... * Merging... * Merged 22577 address ranges.
Removed 22577 items from 223673. Don't know it that is many or not.
Lew - 30 Jun 2009 22:40 GMT > * Loaded 223673 addresses > * Sorting... > * Merging... > * Merged 22577 address ranges. > > Removed 22577 items from 223673. Don't know it that is many or not. I would expect, based on the extremely non-hidden documentation of ArrayList, for that to take time roughly
T = (22577 * 223673 / 2) * k
where 'k' is the time to copy an element from one array location to another.
If 'k' is around 100 nanoseconds, T would be around 252 seconds, or somewhat over 4 minutes. A 'k' of 10 ns would require about 25 seconds of removal time.
-- Lew
Donkey Hottie - 30 Jun 2009 22:54 GMT >> * Loaded 223673 addresses >> * Sorting... [quoted text clipped - 15 lines] > seconds, or somewhat over 4 minutes. A 'k' of 10 ns > would require about 25 seconds of removal time. Great ;)
I'm writing a rival to a Windows application, which does that merge at least 5 minutes in a Athlon XP 1900+ machine. My Java implementation does it now in ten seconds in a Pentium Pro 3 machine.
And I though as a C++ programmer (for 10 years) that Java is sluggish..
Eric Sosman - 30 Jun 2009 23:32 GMT >>> * Loaded 223673 addresses >>> * Sorting... [quoted text clipped - 23 lines] > > And I though as a C++ programmer (for 10 years) that Java is sluggish.. As almost always, you get a lot more improvement out of choosing the right data structures and algorithms than out of shaving cycles or even out of choosing implementation language.
You haven't fully explained what you're trying to do, but at a guess you're collecting a big pile of numeric "addresses" from somewhere, sorting them, possibly eliminating duplicates, and finally combining groups of neighboring addresses into "ranges." ArrayList doesn't strike me as a good candidate for the elimination and combining stages, precisely because of the time required to squeeze out the vacated slots. If you *must* use ArrayList (for reasons the sketchy problem description does not reveal), consider working on it from the end toward the beginning instead of from the beginning toward the end. (But since you're removing only 10% of the total, I don't hold out a lot of hope for a huge improvement therefrom.)
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Roedy Green - 01 Jul 2009 00:02 GMT On Tue, 30 Jun 2009 18:32:21 -0400, Eric Sosman <esosman@ieee-dot-org.invalid> wrote, quoted or indirectly quoted someone who said :
> ArrayList doesn't strike me as a good candidate for >the elimination and combining stages, precisely because of the >time required to squeeze out the vacated slots. It is very quick if you create a new array. See http://mindprod.com/products1.html#SORTED for a set of classes for merging and deduping etc. Unfortunately, the classes were written before generics were introduced.
 Signature Roedy Green Canadian Mind Products http://mindprod.com
"Deer hunting would be fine sport, if only the deer had guns." ~ William S. Gilbert of Gilbert and Sullivan
Eric Sosman - 01 Jul 2009 02:05 GMT > On Tue, 30 Jun 2009 18:32:21 -0400, Eric Sosman > <esosman@ieee-dot-org.invalid> wrote, quoted or indirectly quoted [quoted text clipped - 5 lines] > > It is very quick if you create a new array. [...] Assuming the items being removed are fairly numerous. (As it seems they are, although the O.P. let slip this information only in follow-ups and not in the original "removal is futile" post.)
If only a few items are being deleted, especially if they are known to appear near the end, the work involved in sliding all their successors one place to the left may well be less than the work of copying "all except" to an entirely new array. Horses for courses, and don't blame the horse if he falters when you ride him up the ski jump.
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Arne Vajhøj - 30 Jun 2009 23:59 GMT >>> * Loaded 223673 addresses >>> * Sorting... [quoted text clipped - 23 lines] > > And I though as a C++ programmer (for 10 years) that Java is sluggish.. I would expect a C++ program using STL vector to have the same behavior.
Arne
Joshua Cranmer - 01 Jul 2009 01:55 GMT > I would expect a C++ program using STL vector to have the same > behavior. He should have used the STL list instead ;-).
 Signature Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth
Arne Vajhøj - 01 Jul 2009 02:36 GMT >> I would expect a C++ program using STL vector to have the same >> behavior. > > He should have used the STL list instead ;-). Yep.
But then LinkedList is the solution in Java.
Arne
Arne Vajhøj - 30 Jun 2009 23:57 GMT >>> Just noticed a 'hidden feature of Java' ;) >>> [quoted text clipped - 23 lines] > > Removed 22577 items from 223673. Don't know it that is many or not. It is many.
If all the items were removed first in the list you would have moved approx. 4.8 billion elements because ArrayList is backed by an array and need to move things around after a delete.
Arne
Patricia Shanahan - 30 Jun 2009 22:16 GMT >> Just noticed a 'hidden feature of Java' ;) >> [quoted text clipped - 15 lines] > an unpleasant experience, it does not follow that brushing your > teeth is futile. Also, if access to a List is dominated by iterators with remove, LinkedList is likely to be more efficient than ArrayList. ArrayList is optimized for indexed access.
Patricia
Donkey Hottie - 30 Jun 2009 22:31 GMT > Also, if access to a List is dominated by iterators with > remove, LinkedList is likely to be more efficient than > ArrayList. ArrayList is optimized for indexed access. Thanks!
I was not aware of that kind of a List being in Java library (thought it would be an Apache solution or something).
LinkedList now belongs to my toolkit allright.
Lew - 01 Jul 2009 01:05 GMT >> Also, if access to a List is dominated by iterators with >> remove, LinkedList is likely to be more efficient than [quoted text clipped - 4 lines] > I was not aware of that kind of a List being in Java library (thought it > would be an Apache solution or something). This is where being a Javadoc junkie pays off.
<http://java.sun.com/javase/6/docs/api/java/util/package-summary.html>
There really is no reason not to be aware of the basic collection classes.
 Signature Lew
Roedy Green - 01 Jul 2009 06:58 GMT >There really is no reason not to be aware of the basic collection classes. For a quick overview of what each Collection is good for, see http://mindprod.com/jgloss/collection.html
 Signature Roedy Green Canadian Mind Products http://mindprod.com
"Deer hunting would be fine sport, if only the deer had guns." ~ William S. Gilbert of Gilbert and Sullivan
Sabine Dinis Blochberger - 03 Jul 2009 12:01 GMT > >> Also, if access to a List is dominated by iterators with > >> remove, LinkedList is likely to be more efficient than [quoted text clipped - 10 lines] > > There really is no reason not to be aware of the basic collection classes. I actually like to also refer to Suns Java Tutorial, because it explains things in "plain english". Sometimes the JavaDoc doesn't give much clue on how or when to use a certain class, although often it will link to the tutorial.
<http://java.sun.com/docs/books/tutorial/>
And to collections <http://java.sun.com/docs/books/tutorial/collections/implementations/index.html>
 Signature Op3racional - www.op3racional.eu --------------------- If you're reading this, you're on Usenet <http://oakroadsystems.com/genl/unice.htm>
markspace - 01 Jul 2009 01:11 GMT >> ArrayList. ArrayList is optimized for indexed access. > > Thanks! > > I was not aware of that kind of a List being in Java library (thought it What?! Are you kidding me? I'm sorry but this is bull****. How the hell can you program in Java at all with out at least bumping into the Collections classes? This is really amazing to me. They're in every tutorial. It's basic algorithms and should be instantly familiar to anyone who made it through their lower division course work.... I'm just aghast. Seriously.
Especially a C++ programmer should be on the lookout for something in a new language to replace the STL, and Collections is a big part of Java's answer to the STL.
Donkey Hottie - 01 Jul 2009 11:40 GMT >>> ArrayList. ArrayList is optimized for indexed access. >> [quoted text clipped - 13 lines] > something in a new language to replace the STL, and > Collections is a big part of Java's answer to the STL. I was aware of the Collections, with ArrayList, but not *LinkedList*. It's a late addition, and I have moved out from Java 1.4.2 only lately... Now using Java 6.
Eric Sosman - 01 Jul 2009 13:15 GMT > I was aware of the Collections, with ArrayList, but not *LinkedList*. > It's a late addition, and I have moved out from Java 1.4.2 only > lately... Now using Java 6. java.util.LinkedList is "since 1.2" in the Javadoc.
 Signature Eric Sosman esosman@ieee-dot-org.invalid
Lew - 01 Jul 2009 13:24 GMT > I was aware of the Collections, with ArrayList, but not *LinkedList*. > It's a late addition, and I have moved out from Java 1.4.2 only > lately... Now using Java 6. Yeah, it wasn't introduced until Java 1.2, ten and a half years ago.
 Signature Lew
Patricia Shanahan - 01 Jul 2009 15:39 GMT >> I was aware of the Collections, with ArrayList, but not *LinkedList*. >> It's a late addition, and I have moved out from Java 1.4.2 only >> lately... Now using Java 6. > > Yeah, it wasn't introduced until Java 1.2, ten and a half years ago. This sort of issue is one of the reasons why I read this newsgroup regularly. It is especially valuable for a solo programmer. I know a lot of things exist because people have asked questions about them, or they have been recommended as solutions.
Patricia
Lew - 01 Jul 2009 17:10 GMT Donkey Hottie wrote:
>>> I was aware of the Collections, with ArrayList, but not *LinkedList*. >>> It's a late addition, and I have moved out from Java 1.4.2 only >>> lately... Now using Java 6. Lew wrote:
>> Yeah, it wasn't introduced until Java 1.2, ten and a half years ago.
> This sort of issue is one of the reasons why I read this newsgroup > regularly. It is especially valuable for a solo programmer. I know a lot > of things exist because people have asked questions about them, or they > have been recommended as solutions. You make a very good point. However, the collections classes are among the Java features that are quite fundamental, and when working with such things as collections one should be in the habit of reviewing the Javadocs to see what exists already. For one accustomed to using the Javadocs it would be quite clear after having been alerted to the existence of a decade-old API that the class is not new, even if they didn't discover the class until having been alerted to it.
Usenet is useful, but not the best as the primary source of Java knowledge, let alone the only one. To achieve competence in Java programming, let alone virtuosity, one must be in the habit of reading the Javadocs, the tutorials, search engine results, and yes, even books. It helps to be aware of third-party library sources, too, like Apache Commons (which has even more collections classes), sourceforge and the like.
LinkedList, HashMap, java.util.Collections and other such have been officially part of the Java API since 1998. Like, say, large parts of the java.io and javax.swing packages, there's precious little reason for a professional Java programmer not to be at least aware of their existence, nor to think that they're brand new.
-- Lew
Knute Johnson - 01 Jul 2009 23:19 GMT > Donkey Hottie wrote: >>>> I was aware of the Collections, with ArrayList, but not *LinkedList*. [quoted text clipped - 34 lines] > -- > Lew I'm with Patricia on this. I work by myself and while I read books, the docs and the tutorials, the API is staggering in its size and complexity. The news groups have been invaluable for me in my work. On the Collections issue specifically, I've used them for years but I don't know all the ins and outs of what is more efficient for a particular type of operation. This thread gave me a lot of valuable information that I will use in the future and now to look at code already written with an eye to better performance.
I am not however making any excuses for the OPs ignorance of the API. You are right about that, he should have some clue. We do get folks in here though that have never written a lick of Java code and need to fix something. I have to fix Perl once in a while (and I need the three books I have for that) and have to learn new languages periodically. Sometimes you can be pretty clueless and a good pointer or two will get you going.
The two main reasons I inhabit these newsgroups, is that I received so much help here when I really needed it and that I learn so many new things by reading the posts. Not as many today as in the beginning but like this post some valuable tidbits here and there.
 Signature Knute Johnson email s/nospam/knute2009/
Lew - 30 Jun 2009 22:10 GMT > Just noticed a 'hidden feature of Java' ;) > [quoted text clipped - 4 lines] > > So it seems. removal is futile. Not so hidden, really.
<http://java.sun.com/javase/6/docs/api/java/util/ArrayList.html>
> The size, isEmpty, get, set, iterator, and listIterator operations run > in constant time. The add operation runs in amortized constant time, that > is, adding n elements requires O(n) time. All of the other operations run > in linear time (roughly speaking). That's for *each* removal.
That's because 'ArrayList#remove()' (through an iterator or otherwise)
> [s]hifts any subsequent elements to the left (subtracts one from their indices). <http://java.sun.com/javase/6/docs/api/java/util/ArrayList.html#remove (int)>
The documentation for the collection classes usually indicates the big- O time of the common operations.
I would expect copying to take longer, however, depending on where one is in the iteration. Perhaps there's extra machinery in the iterator logic to normalize the iterator's position within the list.
Or are you copying individual elements as you iterate, skipping over the "removed" ones? That should be much faster than multiple removals.
-- Lew
Donkey Hottie - 30 Jun 2009 22:23 GMT .
> I would expect copying to take longer, however, depending > on where one [quoted text clipped - 8 lines] > multiple > removals. Well, after I discovered the slow speed of removal, I rewrote the method now creates a new List and always just adds the suitable elements and replaces the List instance.
10% of the original List seem to be removed, and it is 1000 times faster now.
Knute Johnson - 30 Jun 2009 23:13 GMT > . >> [quoted text clipped - 17 lines] > 10% of the original List seem to be removed, and it is 1000 times faster > now. I'd be really curious to know if using the LinkedList is significantly faster than ArrayList. If you try it, please post back.
Thanks,
 Signature Knute Johnson email s/nospam/knute2009/
Arne Vajhøj - 30 Jun 2009 23:57 GMT >> I would expect copying to take longer, however, depending >> on where one [quoted text clipped - 15 lines] > 10% of the original List seem to be removed, and it is 1000 times faster > now. Most likely it would be better with another data type than ArrayList.
Arne
Kevin McMurtrie - 01 Jul 2009 06:03 GMT > Just noticed a 'hidden feature of Java' ;) > [quoted text clipped - 4 lines] > > So it seems. removal is futile. Skipped school? This is in the basics of information science. It has nothing to do with Java.
When I think of hidden Java features I think of secret accessor methods, generics insanity, -XX switches, FinalReference, and in-place math operators lacking narrowing checks.
 Signature I will not see your reply if you use Google.
Lew - 01 Jul 2009 13:26 GMT > generics insanity, To what insanity do you refer?
 Signature Lew
charlesbos73 - 01 Jul 2009 16:53 GMT > > generics insanity, > > To what insanity do you refer? erasure? (say as opposed to the way they're done in C#), making them useless from an OO point of view?
public interface UselessGenerics extends Handable<A>, Handable<B> {
"Duplicate class: '.....Handable'
AAAAAAlllllrrrriiiigggghhhhhttttt.
It's just like the broken Java equals() concept: it "works" only as long as you're doing procedural programming.
As soon as you want to some real OOA/OOD->OOP translation you're out of luck.
But I'll give it to you: for 99.9% of all the Java programs out there are just glorified procedural programs it's no big deal.
Mayeul - 01 Jul 2009 17:16 GMT >>> generics insanity, >> To what insanity do you refer? > > erasure? (say as opposed to the way they're done in C#), Point taken I guess. No erasure would have horribly broken Java 1.4 code and previous, so it is necessary. Still, it is a flaw.
> making them useless from an OO point of view? Seems like an overstatement. The commodity of generics fairly facilitates OO design, which is just as valid as an 'OO point of view'.
> public interface UselessGenerics extends Handable<A>, Handable<B> { > > "Duplicate class: '.....Handable' > > AAAAAAlllllrrrriiiigggghhhhhttttt. Just because this doesn't work, doesn't make everything useless.
> It's just like the broken Java equals() concept: it "works" only > as long as you're doing procedural programming. It's not the first time you criticized Java Object.equals(), and you've been asked for clarifications, but I didn't see them. I still fail to understand how not OO or how anti-OO that is.
> As soon as you want to some real OOA/OOD->OOP translation you're > out of luck. [Citation needed]
> But I'll give it to you: for 99.9% of all the Java programs out > there are just glorified procedural programs it's no big deal. Actually that might be true, considering that the vast majority of programming done on the whole world is not done by skilled software designers.
Same for just about any well-known language, I guess.
-- Mayeul
Lew - 01 Jul 2009 18:48 GMT charlesbos73 wrote:
>> erasure? (say as opposed to the way they're done in C#),
> Point taken I guess. No erasure would have horribly broken Java 1.4 code > and previous, so it is necessary. Still, it is a flaw. While there are definitely times when having generics information at run time would help, the fact that it's compile-time only does benefit in that it forces one to handle type issues at compile time, when bugs are much cheaper to fix than at run time. In practice, the need for run-time generification is only for corner cases; most of the time it's completely sufficient to have only compile-time checking.
People misunderstand the generics mechanism. Generics support type analysis, and type analysis can be quite tricky. Hence, use of generics can be tricky. However, once you get all the generics compiler messages ironed out, you have a self-documented ironclad type contract in your code. It's well worth the effort.
charlesbos73 wrote:
>> It's just like the broken Java equals() concept: it "works" only >> as long as you're doing procedural programming.
> It's not the first time you criticized Java Object.equals(), and you've And probably not the last.
> been asked for clarifications, but I didn't see them. I still fail to > understand how not OO or how anti-OO that is. He's just being trollish. Making inflammatory comments without evidence or logic is one of those classic trollish Usenet behaviors.
Others have responded to his criticism that Java is not object- oriented, which, of course, it is, and "charlesbos73" was equally silent regarding their responses. At any rate, I haven't seen anyone respond to any of his responses. For example, his misrepresentation of statements from /Effective Java/ was called to account.
Polymorphism is one of Grady Booch's six pillars of object orientation - by that standard 'Object.equals()' definitely fits the O-O paradigm.
I see no way in which Java violates object orientation except perhaps the support for static members. Java is clearly an object-oriented language. (OK, Smalltalk fans, settle down.)
-- Lew
Eric Sosman - 01 Jul 2009 19:31 GMT > charlesbos73 wrote: >>> erasure? (say as opposed to the way they're done in C#), [quoted text clipped - 6 lines] > in that it forces one to handle type issues at compile time, when bugs > are much cheaper to fix than at run time. Yes, compile-time checking is Good. But it doesn't follow that run-time checking in addition to compile-time checking would be Bad.
> In practice, the need for > run-time generification is only for corner cases; most of the time > it's completely sufficient to have only compile-time checking. Some of the corners are fairly gently rounded, as witness the number of questions that arise here about them. Perhaps the two most common "smooth corners" are constructing instances of and making arrays of the parameterized type:
class Generic<T> { T instance = new T(); // no good T[] array = new T[42]; // no good }
> People misunderstand the generics mechanism. Generics support type > analysis, and type analysis can be quite tricky. Hence, use of > generics can be tricky. A-men, brother! I recall that when generics first came along I decided I'd retrofit them into some of my existing code that used Object references and a lot of casts. NetBeans 3.mumble kept drawing squiggly red lines under usages I thought were valid, and I simply couldn't comprehend what was wrong. I tried a javac compilation from the command line in the faint hope of getting a more comprehensible diagnostic, and lo! javac accepted my code happily!
NetBeans 5.mumble continued to protest my acceptable-to-javac code, and it wasn't until NB 6 came along that the squiggly lines finally went away. If even the tools developers can't get generics right, what hope is there for mere mortals?
> I see no way in which Java violates object orientation except perhaps > the support for static members. [...] Some have disparaged its use of non-objectified primitive types. Zealotry is its own reward, I guess.
 Signature Eric.Sosman@sun.com
Arved Sandstrom - 01 Jul 2009 20:41 GMT [ SNIP ]
> Some of the corners are fairly gently rounded, as witness the > number of questions that arise here about them. Perhaps the two [quoted text clipped - 5 lines] > T[] array = new T[42]; // no good > } [ SNIP ]
About the closest you can get is to use java.lang.reflect.Array.newInstance(Class<?>, int) or java.lang.reflect.Array.newInstance(Class<?>, int[]). It's not perfect, as pointed out by Goetz(http://www.ibm.com/developerworks/java/library/j-jtp01255.html, about halfway down the page).
AHS
Kevin McMurtrie - 02 Jul 2009 07:59 GMT > > generics insanity, > > To what insanity do you refer? Simple generics work fine but all hell breaks loose on complex data. The solution just isn't very robust. This sums it up well: http://www.ibm.com/developerworks/java/library/j-jtp01255.html
Don't even get me started on ReferenceQueue generics.
It also forgets to mention that each genericized method has a non-genericized method automatically created by the source compiler:
class Foo extends HashSet<Integer>{} class Bar<T> extends HashMap<Integer, T>{} class Moo<T> extends HashMap<Integer, T> { @Override public T put(Integer key, T value) { return super.put(key, value); } }
new Foo().add("hello"); //Compile Error new Bar<String>().put("hello", "there"); //Compile Error new Bar().put("hello", "there"); //OK... until later.
//Drumroll.... new Moo().put("hello", "there"); //ClassCastException on a bogus line number
 Signature I will not see your reply if you use Google.
Lew - 02 Jul 2009 12:55 GMT >>> generics insanity, Lew wrote:
>> To what insanity do you refer?
> Simple generics work fine but all hell breaks loose on complex data. The > solution just isn't very robust. This sums it up well: > http://www.ibm.com/developerworks/java/library/j-jtp01255.html I don't see insanity here.
> Don't even get me started on ReferenceQueue generics. > > It also forgets to mention that each genericized method has a > non-genericized method automatically created by the source compiler: The output of the compiler is generics-free, of course. This is well documented.
> class Foo extends HashSet<Integer>{} > class Bar<T> extends HashMap<Integer, T>{} [quoted text clipped - 7 lines] > > new Foo().add("hello"); //Compile Error Duhhh. That's the purpose of generics.
> new Bar<String>().put("hello", "there"); //Compile Error Duhhh. That's the purpose of generics.
> new Bar().put("hello", "there"); //OK... until later. > > //Drumroll.... > new Moo().put("hello", "there"); > //ClassCastException on a bogus line number Well, duhhh. You used a raw type, which does create a compiler warning. One is thoroughly advised not to mix raw types and generics. That ClassCastException is exactly what one would have gotten before generics were added to Java.
What you illustrate here is not generics insanity but a typical programmer error.
 Signature Lew
Kevin McMurtrie - 02 Jul 2009 16:07 GMT > >>> generics insanity, > [quoted text clipped - 46 lines] > What you illustrate here is not generics insanity but a typical programmer > error. The last two have been problems when updating existing code in a large project. Regardless of whether classes Moo or Bar are constructed with generics, the Map key is clearly defined as Integer. The source compiler added a hidden method that allows compilation of code that can not run.
Compiler warnings are great for new code, but not so great for the millions of lines of old code.
 Signature I will not see your reply if you use Google.
Peter Duniho - 02 Jul 2009 17:43 GMT > [...] > Compiler warnings are great for new code, but not so great for the > millions of lines of old code. I'd say they're pretty good for old code, even millions of lines, if you actually go _fix_ them.
Sheer quantity of code isn't really a justification for leaving errors in it.
Kevin McMurtrie - 03 Jul 2009 06:37 GMT > > [...] > > Compiler warnings are great for new code, but not so great for the [quoted text clipped - 5 lines] > Sheer quantity of code isn't really a justification for leaving errors in > it. Tell your boss you want to spend a full year fixing Java 1.5 warnings because there's a 1 in 600 chance that each is a bug. Next convince your boss that the risk in fixing them is lower than the original 1 in 600 odds.
 Signature I will not see your reply if you use Google.
Lew - 03 Jul 2009 06:49 GMT Peter Duniho wrote:
>> Sheer quantity of code isn't really a justification for leaving errors in >> it.
> Tell your boss you want to spend a full year fixing Java 1.5 warnings > because there's a 1 in 600 chance that each is a bug. Next convince > your boss that the risk in fixing them is lower than the original 1 in > 600 odds. Oh, yeah, leave the bugs in the code. That's a *real* professional attitude!
 Signature Lew
Patricia Shanahan - 03 Jul 2009 14:08 GMT > Peter Duniho wrote: >>> Sheer quantity of code isn't really a justification for leaving [quoted text clipped - 7 lines] > Oh, yeah, leave the bugs in the code. That's a *real* professional > attitude! I agree that it is a really professional attitude. *Every* change to code carries a risk of creating a bug. Clean existing code and a good development process reduce the risk, but it cannot be reduced to zero. Every change also has a cost in programmer time and effort that could have been spent on something else.
I've seen important releases, with real bug fixes and features the users wanted, delayed because someone decided to clean up a harmless departure from technical desirability, and introduced a new bug in the process.
A professional programmer should step beyond purely technical issues to think about whether the risk and cost associated with a change are justified by the benefit.
I do not mean to imply one should never clean up warnings etc. I strongly prefer warning-free code. Neither the cost nor the benefit of clean-up is zero, so it is a non-trivial decision.
Patricia
Arved Sandstrom - 03 Jul 2009 15:08 GMT >> Peter Duniho wrote: >>>> Sheer quantity of code isn't really a justification for leaving [quoted text clipped - 27 lines] > > Patricia One thing I learned early in my career, which is related to this subject, is, as much as it pains you to see spelling errors in method names and variable names (for example, coders using "principal" when they meant "principle", and vice versa, or "similiar" instead of the correct "similar"), don't take advantage of your IDE's refactoring capability to correct everything. :-) It's amazing how badly this can bite you.
In the larger sense I agree with Patricia. Once a piece of code is baselined, you can see an obvious error staring you right in the face while working on something else, but you have to resist the temptation to fix it. Code somewhere else may rely on that error. You need to have permission to make the fix, after explaining why you should fix it.
AHS
Peter Duniho - 03 Jul 2009 16:55 GMT > [...] > In the larger sense I agree with Patricia. Once a piece of code is > baselined, you can see an obvious error staring you right in the face > while working on something else, but you have to resist the temptation > to fix it. Code somewhere else may rely on that error. You need to have > permission to make the fix, after explaining why you should fix it. The points you and Patricia both make are absolutely correct. One should never approach changes to code lightly.
But, that doesn't mean there's no value in deciding to spend the time to tackle warnings in the code.
(Of course, this entire discussion assumes a code base that somehow managed to get to "millions of lines of old code" without it _ever_ occurring to someone that they ought to be compiling with warnings-as-errors, or otherwise requiring the code be warning-free. I suppose in that environment, it might actually be a better move for a competent programmer to just find a job at a place that doesn't suck so much).
Pete
Lew - 03 Jul 2009 17:14 GMT >> Peter Duniho wrote: >>>> Sheer quantity of code isn't really a justification for leaving [quoted text clipped - 25 lines] > strongly prefer warning-free code. Neither the cost nor the benefit of > clean-up is zero, so it is a non-trivial decision. What you suggest is valid and certainly not using "sheer quantity of code" as a justification. However, the attitude that "mere" warnings are acceptable does lead to real bugs that cause trouble in production. Where did Kevin's "1 in 600 chance" assessment come from? Statistical analysis? I feel certain not, rather, it was pulled from a particular posterior region.
99% (another posteriorily postulated probability) of the time someone recommends simply not fixing warnings they are ignoring real bugs.
I've seen this in my current large-team project and in past such projects, where a request for customer A's data resulted in data from customer B's records because people ignored "trivial" warnings, or merely left out simple code analysis that would have revealed trouble that didn't raise a compiler warning.
A professional programmer should step beyond purely arbitrary (and specious) characterizations of risk and cost associated with a change to understand that *every* failure to change code that has warnings carries a risk of perpetuating a bug. I agree that risk cannot be reduced to zero, but leaving known warnings in code increases that risk.
Code should not be released into the code base in the first place with warnings showing. If that's done, then the question of the risk of change becomes moot, because there's nothing to change. When that principle is ignored, the technical debt mounts until people can aver that the "risk of cleaning up" is too high to shoulder. In practice, I've seen much more failure to perform correctly arise from leaving known bugs in than from cleaning them up.
Once they're in, then the cost of repair is high, but there are mitigation strategies even so. Again using my current project as an example, where we have multiple warnings (from the compiler and from Findbugs, a /sine qua non/ for robust code) the reviewers recommend to management a strategy that cleans up the most critical bugs right away, but defers others to a refactoring timetable that also improves fundamental algorithms or updates to later features (e.g., JPA over old-style ORM code). Combination strategies like fixing the algorithm often reap much larger benefits than simple patches for similar cost, including simplification of the code to reduce the risk of introducing new bugs.
Just because one maintainer was careless with a fix doesn't mean that the fix itself was a bad idea. Zero defects should be the goal rather than resignation to poor quality. You can't use "it's impossible" as an excuse not to aim for it. And you most certainly can't use arbitrary and unsupported fake statistics to support a cost/benefit analysis.
 Signature Lew
Peter Duniho - 03 Jul 2009 16:51 GMT > [...] >> Sheer quantity of code isn't really a justification for leaving errors [quoted text clipped - 3 lines] > Tell your boss you want to spend a full year fixing Java 1.5 warnings > because there's a 1 in 600 chance that each is a bug. Been there, done that. We fixed the warnings. If it takes a full year, then your boss may want to put more than one guy on the project.
> Next convince > your boss that the risk in fixing them is lower than the original 1 in > 600 odds. I didn't waste any time making up numbers like "1 in 600 chance" or "lower than the original". Complete non-issue.
Pete
Lew - 02 Jul 2009 22:30 GMT >>> class Moo<T> extends HashMap<Integer, T> >>> { [quoted text clipped - 3 lines] >>> } >>> } ...
>>> new Moo().put("hello", "there"); >>> //ClassCastException on a bogus line number ...
> Regardless of whether classes Moo or Bar are constructed with > generics, the Map key is clearly defined as Integer. The source > compiler added a hidden method that allows compilation of code that can > not run. It's not a "hidden" method, it's 'public Object put( Object key, Object value )', as is always the case when you erase generics, and is heavily documented as such.
The 'Map' key is based on extending 'HashMap<Integer, T>', which erases to 'HashMap <Object, Object>', and is therefore not "clearly defined as Integer" at run time. You call this "insanity", but it's really just plain old simple erasure. This is the common programmer error to which I alluded upthread, and the inevitable consequence of mixing raw and generics types. You insert a 'String' key into the map, then try to extract it to an 'Integer' target, thus yielding the 'ClassCastException', just as documented and expected for this scenario. Again, this is what would happen if you tried this prior to Java 5.
It makes no sense to break the rules, get the exact result that is documented for breaking the rules, then complain that you get that result.
 Signature Lew
Tom Anderson - 01 Jul 2009 23:16 GMT >> Just noticed a 'hidden feature of Java' ;) > > When I think of hidden Java features I think of secret accessor methods, > generics insanity, -XX switches, FinalReference, and in-place math > operators lacking narrowing checks. FinalReference is new to me. Now i know what it is, but that's it - what can you do with it?
tom
 Signature How's it going to end?
Kevin McMurtrie - 02 Jul 2009 07:42 GMT > >> Just noticed a 'hidden feature of Java' ;) > > [quoted text clipped - 6 lines] > > tom Try to avoid them! When you override finalize(), each instantiation of your object creates a Finalizer that's a subclass of FinalReference. A Thread polls its ReferenceQueue and calls to finalize() during the GC process. Overriding finalize(), even with an empty method, on frequently created objects can be a catastrophic drain on performance and memory.
 Signature I will not see your reply if you use Google.
Lew - 02 Jul 2009 13:00 GMT >>>> Just noticed a 'hidden feature of Java' ;) >>> When I think of hidden Java features I think of secret accessor methods, [quoted text clipped - 9 lines] > frequently created objects can be a catastrophic drain on performance > and memory. Could someone please point to a FinalReference reference?
 Signature Lew
Arved Sandstrom - 02 Jul 2009 14:19 GMT >>>>> Just noticed a 'hidden feature of Java' ;) >>>> When I think of hidden Java features I think of secret accessor [quoted text clipped - 11 lines] > > Could someone please point to a FinalReference reference? I couldn't find one myself. Looking at the code for java.lang.ref.Finalizer, which extends java.lang.ref.FinalReference, is probably the best way to figure it out.
AHS
Lew - 02 Jul 2009 22:32 GMT Lew wrote:
>> Could someone please point to a FinalReference reference?
> I couldn't find one myself. Looking at the code for > java.lang.ref.Finalizer, which extends java.lang.ref.FinalReference, is > probably the best way to figure it out. So, once again, not a hidden *feature* but a hidden *implementation detail*.
I thought one of the tenets of O-O programming was to hide implementation details.
 Signature Lew
Arved Sandstrom - 03 Jul 2009 00:34 GMT > Lew wrote: >>> Could someone please point to a FinalReference reference? [quoted text clipped - 8 lines] > I thought one of the tenets of O-O programming was to hide > implementation details. Both Finalizer and FinalReference are package private in java.lang.ref, so pretty clearly these are not classes that the masses are meant to use. In other words, definitely implementation details.
AHS
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|