Java Forum / First Aid / November 2005
Junit newbie
Jason - 18 Nov 2005 00:33 GMT I'm not getting junit at all.
Is the general strategy to write trivial tests, boundary tests, and perhaps a "range of data" test? It seems like I can easily spend a 3:1 ratio of time in junit versus my actual class. And testing would still be kicking around in the dark.
Let's say I'm writing a compression class similar to a stream (because I am) which only reads and writes byte arrays. What would the minimum testing include? How about:
de/compress a null iteratively test de/compressing a single byte array with the values ranging from 0 to 0xFF test de/compressing some very large byte array (from a file 2GB file perhaps) test de/compressing some random data
As for the last one, I'm not even sure that's useful as the test conditions that caused the failure would be squirrely. The condition would almost certainly rely on the previous compression attempt and that debugging info will be lost before the test fails.
Can someone take pity an give me the big picture on this stuff?
Thanks! Jason
Darryl L. Pierce - 18 Nov 2005 03:29 GMT > I'm not getting junit at all. > > Is the general strategy to write trivial tests, boundary tests, and perhaps > a "range of data" test? It seems like I can easily spend a 3:1 ratio of > time in junit versus my actual class. And testing would still be kicking > around in the dark. Yep, that's quite likely and not necessarily a bad thing. Development done correctly should only be a small portion of the overall project time, and unit testing should be at least as long as development. Spending three times as long writing the tests as the code ensures you don't spend three or four times as long debugging problems in the code...
 Signature Darryl L. Pierce <mcpierce@gmail.com> Homepage: http://mcpierce.multiply.com/ "Bury me next to my wife. Nothing too fancy, though..." - Ulysses S. Grant
"." - 21 Nov 2005 19:07 GMT > I'm not getting junit at all. > [quoted text clipped - 20 lines] > > Can someone take pity an give me the big picture on this stuff? I've seen this a lot in classes I taught. A portion of the assignment goes towards testing the application. Some students will write a 50 line application and hand in, literally, 30 pages of test output.
You can narrow the testing down a little by thinking about how things can be grouped. If the start of a method begins with an:
if (case 1 is true) { // something } else if (case 2 is true) { // something else } else // case 3 is true) { // whatever is left }
I'm going to need at least 3 tests in jUnit.
If you have been programming for a while you noticed that at first you'd right out 10 different if/elseif/else statement. As you refine the program you realize that the solution for case 1, 3 and 7 is the same so you combine them. This forms a compound boolean statement. There might be multiple ways into the statement. Some will say you need 3 test cases for the 1 boolean statement. The logic is that if someone messes up part of the statement you might catch it. Others say that you only need 1 test case because all 3 will exercise the same body; i.e. they are equivalent test cases so you only need to pick one. Having 3 test cases for the 1 statement is safer. Is your boss willing to invest the time to code and maintain them all?
As to what if test2 requires test1 to pass? I like to set up my tests is the setup() function provides everything necessary for the test to proceed. Therefore, the setup() for test2 is to ensure that the output for test1 exists.
Another option is to map the dependencies and have test2 check if test1 failed. If test1 fails, don't even bother running test2. Or have an assert at the beginning of test2 that checks for the success of test1 and outputs good information in the log, i.e. "test2: failed. Relied on test1, which failed."
I like to do the latter. That way if 3047 tests rely directly or indirectly on test1 and test1 fails. The other 3047 tests are unknown. Unknown is not a failure but it is a potential failure; could impact schedule.
Finally, the de/compressing random data is a potentially good test. You want to test that your code does what it is supposed to do but you also want to test that your code does not do what it is not supposed to do. For example, if you give it bad data it should die gracefully. It should not format the hard drive, corrupt data or reboot your computer. It should recover nicely.
 Signature Send e-mail to: darrell dot grainger at utoronto dot ca
Oliver Wong - 21 Nov 2005 20:10 GMT > I'm not getting junit at all. > > Is the general strategy to write trivial tests, boundary tests, and > perhaps a "range of data" test? One strategy is to write tests until you don't feel like you're writing useful tests anymore.
Another strategy is to imagine yourself as being a "lead programmer" to has to delegate work to "junior programmers". You assume that the junior programmer has a vague idea of what the method (s)he has to code is supposed to do, but doesn't know the details (e.g. what should happen at the boundaries, like you mentioned above). Write enough tests so that the junior programmer can be relatively confident that his/her code works, without coming back to you to ask for clarifications.
Also, while coding, if you notice a bug, it's usually a good idea to write a test which will detect that specific bug so that it doesn't slip by again.
It might also be worth considering both "black box testing" and "glass box testing". In the former (black box), you don't assume anything about the implementation. You write tests that make sure the method does what it's supposed to do, and that's it. In "glass box testing", you know what the algorithm is that implementing the feature you're testing, so you'll write tests which will ensure every line of code gets executed at least once (e.g. make sure that both the "true" and "false" paths of every if statement are taken).
Usually you'll want to have a good mix of black box and glass box testing.
> It seems like I can easily spend a 3:1 ratio of time in junit versus my > actual class. It's not unusual in test-driven development for one to spend more time writing tests than writing code.
> And testing would still be kicking around in the dark. > [quoted text clipped - 13 lines] > would almost certainly rely on the previous compression attempt and that > debugging info will be lost before the test fails. Some people feel tests should be fast or else you won't want to actually run the test. If you agree with this philosophy (I don't), then you probably want to avoid the 2GB test file as that'll probably be a long test.
Some people have configured their IDEs to automatically run unit tests immediately after a compilation, so that the failed tests actually show up in the error logs (e.g. as compilation errors or warnings).
- Oliver
Monique Y. Mudama - 21 Nov 2005 21:55 GMT > It's not unusual in test-driven development for one to spend > more time writing tests than writing code. A view I've heard espoused in test-driven development is that you write tests first. When all your tests pass, your code is done. If you nevertheless find your code misbehaving, you write a new test or fix the existing broken one. Again, when your code passes the new test, it is done.
It's an attractive strategy that I've never had opportunity to use.
 Signature monique
Ask smart questions, get good answers: http://www.catb.org/~esr/faqs/smart-questions.html
Oliver Wong - 21 Nov 2005 22:37 GMT >> It's not unusual in test-driven development for one to spend >> more time writing tests than writing code. [quoted text clipped - 6 lines] > > It's an attractive strategy that I've never had opportunity to use. There's a huge psychological barrier. While 1/3 of the way through writing the tests, I can't help but think "If I had just wrote the code directly, I would have been finished by now."
On the other hand, there's a huge satisfaction to seeing that little green bar fill up indicating that your code passed all the tests. (Less satisfaction when your screen fills up with red text indicating that some or all of your tests blew up).
For the COBOL Interpreter I'm working on, we've got a suite of about 450 COBOL programs, and we've set up JUnit tests to automatically try to parse, pretty print, and run them all one after another (so about 1350 test runs). Some programs are intended to produce a certain output, while other programs are intentionally crafted to generate specific compilation-time errors.
Imagine trying to run these 1350 by hand, and then actually expecting the output to see if it matches the expected output! With JUnit, the whole thing takes 7 to 10 minutes, and can be a real confidence booster when you make a change and you're not sure what (if anything) those changes are gonna break.
- Oliver
Andrew McDonagh - 21 Nov 2005 23:17 GMT >>> It's not unusual in test-driven development for one to spend >>> more time writing tests than writing code. [quoted text clipped - 10 lines] > writing the tests, I can't help but think "If I had just wrote the code > directly, I would have been finished by now." Sure, but then like any other design technique, we developers always struggle with that one. Like you report though, TDD whilst being a Design Methodology first, serves us well as a regression test suite too.
> On the other hand, there's a huge satisfaction to seeing that little > green bar fill up indicating that your code passed all the tests. (Less [quoted text clipped - 14 lines] > > - Oliver Andrew McDonagh - 21 Nov 2005 23:15 GMT >> It's not unusual in test-driven development for one to spend >> more time writing tests than writing code. [quoted text clipped - 6 lines] > > It's an attractive strategy that I've never had opportunity to use. You have heard correctly, but have missed one important step of the 3 step process:- Red, Green, Refactor.
Red - Write a Failing test.
It should fail because, the production code it tests hasn't been written - only the stubs. A vital to ensure the test is working as you expect - its easy to write a test that passes accidentally.
Green - Make the test Pass.
Write JustEnough to make the test pass - no more. Just enough can be to return 'true', 42, fish, etc depending upon what the test is asserting. The next test will make us remove the hard coded value, to us what ever appropriate (and minimum) implementation.
Refactor - Remove Duplication - The Most important step
Because we write JustEnough code to make a test pass and no more, we usually end up with a design that isn't optimal from a design point of view. Again, the change we do should be minimal, but normally results in a design that is very intent revealing and properly normalized (As in Single Responsibility Principle, Law of Demeter, etc).
Note, its also one test at a time - always!
It isn't until you have been trying this for awhile, that you begin to see what TestDrivenDevelopment is actually about - namely that its a Design methodology, not a testing methodology.
There's a group of people working on JBehave as a replacement for JUnit, because they feel so strongly that the use of the word 'test' within the framework and the methodology name, is wrong. They want to emphasize the Design or Behavior Specification that is at the heart of TDD.
The unit tests of TDD are essentially the executable form of a design, much like UML diagrams are a pictorial form of a design. The argument is that executable forms are more likely to be kept up to date (cause they are part of the build system) and that they force a better design: Low coupling, High cohesion. The force is felt when we write the test, if its difficult to write, or test, or setup the test data, its telling us we have a problem with coupling.
Andrew
Monique Y. Mudama - 21 Nov 2005 23:24 GMT >>> It's not unusual in test-driven development for one to spend >>> more time writing tests than writing code. [quoted text clipped - 31 lines] > normalized (As in Single Responsibility Principle, Law of Demeter, > etc). Well, I mostly read about test driven design in "diving into python", and the author insisted on stopping once the code works. He argues that otherwise, we, being perfectionists, would fiddle with the code forever.
I tend to agree with you that refactoring is important for anything but throwaway code (and throwaway code usually isn't).
 Signature monique
Ask smart questions, get good answers: http://www.catb.org/~esr/faqs/smart-questions.html
Oliver Wong - 22 Nov 2005 15:30 GMT > Red - Write a Failing test. > > It should fail because, the production code it tests hasn't been written - > only the stubs. A vital to ensure the test is working as you expect - its > easy to write a test that passes accidentally. I've often wondered about the "make sure the test fails". There might exists functions which, given the correct situation, "do nothing" (e.g. removeAllOddNumbers() on a list of only even numbers), and so when a stub is written (which does nothing), the test that ensures that nothing happens will seem to succeed at first. It might still be a useful test, because as more code gets written, perhaps "nothing" will no longer happen.
- Oliver
Monique Y. Mudama - 22 Nov 2005 17:59 GMT >> Red - Write a Failing test. >> [quoted text clipped - 9 lines] > at first. It might still be a useful test, because as more code > gets written, perhaps "nothing" will no longer happen. Yeah, I don't think it would fail in that case. But overall the test suite *would* fail, because you'd also have a test in which you provided a list of even and odd numbers, and that one would fail.
 Signature monique
Ask smart questions, get good answers: http://www.catb.org/~esr/faqs/smart-questions.html
Andrew McDonagh - 27 Nov 2005 23:34 GMT >>Red - Write a Failing test. >> [quoted text clipped - 10 lines] > > - Oliver Yes these kind of starting points can be tricky to get straight in your mind. But way start with a list of just even?
------------------------------------
Test One: testListOfOddsReturnsEmptyList() {...}
Which gives us:
int[] removeAllOddNumbers(int[] numbers) { return new int[0]; }
------------------------------------
Test Two: testListOfEvensReturnedUnchanged() {...}
Which gives us:
(remember we currently only have two tests: All odds & All Evens!)
int[] removeAllOddNumbers(int[] numbers) {
if (numbers[0] % 2 = 0) // Assume all numbers even so just return them. return numbers; else //Assume all numbers odd, so return new list return new int[0]; }
------------------------------------
Now, at this point, if all our method needs to do is differentiate between lists of the modulus (even or odd) then we are finished. But, we want our method to do more than this, so we continue.....
Test Three: testMixedModulusListShouldHaveOddsRemoved() {...}
Which gives us:
int[] removeAllOddNumbers(int[] numbers) { int[] justEvenNumbers = new int[numbers.length];
int evensIndex = 0; for (int index = 0; index < numbers.length; index++) { // if even number if (numbers[index] % 2 = 0) { justEvenNumbers[evensIndex] = numbers[index]; }
}
return justEvenNumbers; }
---------------------------------
So you see, depending upon where we start, we implement something different.
That being said, there's been occassions where I've been stumped in ways to get the test failing first because I've already implemented enough code to make it pass. In these circumstances,I have a few choices:
1) Live with the tests JustPassing (unlikely!)
2) deliberately noble the production code so as to see the test fail - useful for validating the test - and have caught a few bugs within the test with this approach.
3) realised the problem isn't the production code already doing enough, its the tests not being well factored - they need refactoring. As in, the new test is probably duplicating nearly everything of another test and so its logic has already been validated, its only the test data and expected results that has changed - so refactor the tests to remove this. (this is a good state, as it shows the production codes design is starting to show OpenClosedPrinciple characteristics.)
4)....
hth
Andrew
Darryl L. Pierce - 23 Nov 2005 11:18 GMT > A view I've heard espoused in test-driven development is that you > write tests first. When all your tests pass, your code is done. If [quoted text clipped - 3 lines] > > It's an attractive strategy that I've never had opportunity to use. I've been using this strategy quite a bit in my current project. It has resulted in some of the most stable code I've seen produced on a first pass. And, as Oliver said, when bugs are found, new test cases are written and the code fixed until those tests pass. And we have a guarantee that, when we modify the system, we haven't broken anything if the unit tests succeed.
To compliment this system, we also have a continuous integration environment where every checkin results in the code being exported and unit tests run on the repository server. If the build fails (or if the checkin fixes a previous build failure) then all of us are emailed about the failure or fix. And, if all unit tests succeed then our test server is automatically updated with the latest code. This latter I am kind of iffy on (I would prefer willful rather than automated test deployments) but it's work so far for us.
All in all, a very productive environment.
 Signature Darryl L. Pierce <mcpierce@gmail.com> Homepage: http://mcpierce.multiply.com/ "Bury me next to my wife. Nothing too fancy, though..." - Ulysses S. Grant
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|