That's a difficult one. Issues with JUnit implementation of the concept,
certainly. I actually think unit testing has a very localised effect on
the running of a project, and needn't impact on other aspects of the
life-cycle. Kind of like having a common set of code formatting conventions.

The only negative thing I can come up with is that writing tests is
really, really boring. Pissing about with debuggers, traces, printfs and
bug reports is much more interesting (true).

Tom Hawtin

Frank Zappa named his daughter Moon Unit. This is a blatant rip off of
the Zappa name.

Curious, why would testing in any form or with any framework  be a
negative thing?

Keep in mind, JUnit (and its other language variants dUnit,cppUnit,
sUnit, NUnit, etc) are just unit testing frameworks and as such can be
used in may ways.

The reason I mention this, is because whilst there is value in unit
testing our code, these particular frameworks have come about to support
TestDrivenDevelopment (TDD).

TDD uses unit tests as a means of capturing the design that we create,
much like UML diagrams capture design.

TDD is a design methodology not a testing one.

The TDD cycle being:

1 Make a Failing test
2 Make the test pass
3 Refactor to remove duplication

However, every time this cycle has been completed, the existing tests
start to also be a regression test suite for the code at the unit level.

TDD's unit tests (or more right -Programmer tests) do not replace
acceptance/integration tests - they support them.

As for JUnit - I have only good things to say about it - I've learnt
more about real OO, design, development & testing in the 4 years I've
been using it, than in the previous 6 years developing.

Andrew

The JUnit approach (that the first failure ends the test) seems unproductive
to me. At times it's apt, but there are other times where I'd like to
continue the test and generate a report of all the failures rather than only
be told about the first one.

I *like* JUnit; I have a great deal of confidence that the code for which I
have JUnit tests works as expected.

Drawbacks to JUnit:

1. It is hard to get an entire project to adopt JUnit if not everyone
believes in automated unit tests.

2. It is sometimes (often, in our case) difficult to write a *unit* test for
each class in isolation.

3. Doing the work to get adequate test coverage from a unit test suite is
generally a thankless job -- unless management buys in to the notion that it
is worthwhile and makes the time to get it done.

4. Maintaining the test suite along with the code it tests is sometimes
painful. (I just changed some code and I have a failing test. Is the test
failing because it's a real failure of the code or because the test case is
now incorrect?)

5. You have to train the people who write the tests what constitutes a good
test. (IE, random or pseudorandom inputs are a bad idea; fixed paths on a
filesystem are a bad idea; etc.)

I wouldn't want to put anyone off the idea of testing, but I do want to raise a
couple of small warning about automated tests.

Don't get me wrong, I /like/ automated tests -- what I want to say is that they
can have a downside too, and that you should be aware of it and compensate
accordingly.

[BTW, nothing in the following is Java-specific, even though I mention some
Java-based tools, they are only named as examples, and I may not even have used
them myself.]

Testing means many different things to different people and in different
contexts.  There are several axes of variation, the two I'm interested in here
are granularity, and attitude.

A wise man once told me that "testing is a deliberate attempt to break the
system".  I think that's a very good description of the attitude that should be
present when testing -- or rather, should be applied during /some/ testing.
And that's what I find is missing in almost any automated testing -- whether
the wildly faddish TDD or classical overnight test suites -- the purpose of the
test is to confirm that the system (or module) works.  There is no active
intent to make the system break, there is no intelligent exploration of
corner-cases, there is no inspired guessing at possible-unanticipated
combinations of inputs.  Above all, there is no exploration of the problem
space -- the suite tests the same combinations each time.  (The big exception
to this -- which I have very rarely seen used in practise -- is when automation
is used for a brute-force, exhaustive, exploration of a significant sub-set of
the problem space).  I like to see some real testing (in the above sense) as
part of the development effort.  To me (and assuming that exhaustive testing is
infeasible) that means interactive testing.  Write a test harness or use
something like BeanShell.  Try things out, did they work ?  Did they work
/exactly/ how you expected ?  Push things a bit.  Did the disk-light flash when
it shouldn't ?  Did the screen flicker more than you expected during your GUIs
repaint ?  Try wildly implausible combinations.  You are trying to break your
own code -- which requires imagination and attention to detail.  Some people
like to step through code under the debugger looking for stuff that "works",
but doesn't take quite the expected path through the cod; that's another
example of the same kind of thinking.

Anyway, the warning I wanted to give about automated testing isn't just that
it's not an adequate substitute for aggressive testing (in the above sense),
but also that there's a risk that it will /displace/ aggressive testing.  Once
the automated tests are written (whether before, along-side, or after the code
they apply to), it takes a great deal of self-discipline[*] not just to rely on
those tests.  Press the button, everything comes up green, "Good, it works!".
That's not /testing/ -- it has a great deal of value, but it's confirming that
the system works on some inputs, not trying to find the inputs that break it.

([*] /I/ don't have that much self-discipline, so -- although I've always put a
great deal of effort into testing -- I've settled into a working pattern where
I don't write automated tests until /after/ I'm satisfied that my stuff works.
So my tests are actually regression tests from the very first, since a newly
written test that fails is, according to my rules, a development failure
(either the test itself is buggy -- happens frequently -- or I'm writing tests
too soon).  Actually, I quite often write brute-force/exhaustive tests at this
point too, and failures there /are/ permitted.)

Now, obviously, it's not a black-and-white situation.  Automated test suites
can attempt to explore large chunks of the problem space -- if enough time is
devoted to it by sufficiently skilled and determined programmers.  (I take it
as obvious that /comprehensive/ test suites are a mere wish-fulfilment
fantasy -- the combinatorial explosion kills the idea stone dead.)  But
elaborate test suites have costs too.  Which brings me to the second point --
granularity.

Code changes.  Right from the start, it changes a lot.  (I'm not talking about
changes in response to changing requirements -- that's a different issue).  IMO
that change is a necessary part of getting a properly working and maintainable
system.  Tests, quite obviously, add "weight" to that, in the sense that any
test for a piece of functionality that is changed will have to change too.  If
you refactor stuff (and you've got any tests written) then you'll have to
refactor them too.  Tests, of any sort, inhibit the natural process of change
as a software design grows -- to take a silly example, there's not much point
in testing the property-file handling you are writing this morning if this
afternoon you are going to rip it out and replace it with Windows registry
lookup...    (Presumably a true TDDer would put that the other way around --
there's not much point in writing code to be tested by your property-file
handling tests if you are just going to rip them out and replace them with
Windows registry handling tests this afternoon ;-)  You have to balance the
cost of testing sub-components against the benefits, taking into account the
near certainty of change.  You can all-but eliminate the costs of testing stuff
that will later be scrapped by concentrating on end-to-end testing, testing a
single business function, rather than the components that interact to perform
it, but then (A) you are only exploring an extremely impoverished slice of the
problem space (not aggressive testing at all) and (B) you don't get the very
real benefits of unit tests.  So you'll probably decide that it's better to
have at least /some/ unit testing, and try to optimise the amount so that it
doesn't add too much weight to your development process.  Clearly, the easier
and simpler it is to invent, implement, and run each test, then more testing
you can do before you pass the break-even point.  And that's where my second
warning comes in -- in my experience, writing scripted tests (JUnit, and
similar) takes a lot of work, about an order of magnitude more work than doing
the same tests in the kind of interactive environment I as talking about
earlier.  So, in my experience, I can do about an order of magnitude /more/
testing of code-still-under-development (without compromising my development
fluidity) by avoiding the formal tools.  The downside is that I do end up
writing scripted tests /as well/ -- but only for the final code, and (as I've
already said) I prefer to use two different styles of testing anyway.

   -- chris

I only write code in my spare time, so take this accordingly.  My main
objection to the JUnit "philosophy" is that it does nothing to make
testing non-observable behavior easier.  Per the JUnit FAQs, they think
that only observable behavior should be unit tested most of the time.  I
simply don't agree with this.