mentis vulgaris
simple thoughts | jason smith
Why Unit Test?
Posted by Jason Smith - 20/01/10 at 10:01:27 amNobody changes until the pain of staying the same becomes greater than the pain of change ~ Anonymous
If the code I come across in my work is any example, most developers haven’t drank the Unit Testing Kool-Aide. Given the old saw about change and pain, I can almost understand. Almost.
So, why should I unit test?
Let’s answer that question by understanding who benefits from them. Just like any other software we write, when we understand who we write our software for, we’re more likely to write software that will serve them best.
Unit Tests are for programmers. If you write code, they are for you.
Big Deal
Sure, unit tests help demonstrate a body of code does what it should. So does white box testing, functional testing, and integration testing. Really, wouldn’t our time be better served stepping through the code for some test cases and a sample app or two, and letting those fiends in QA figure out ways to break our code?
In reality, verifying your code works is just one benefit of unit tests. There are other, further reaching benefits, when you realize the where the bulk of software development cost really lies:
- It documents our code in a way that comments could never hope to match
- It protects dependent code from future breaking changes
- It forces us to decrease coupling (improving our designs – even more so if you take unit testing all the way to TDD)
- Helps us uncover (and prevent) bugs earlier and faster (it beats the pants off of using the debugger to find a break).
Note those benefits are not about insuring the tested code works right now. It’s also about insuring our code won’t break tomorrow. It’s about insuring the newly hired developer understands what the code does so she doesn’t accidentally break it with a “fix”. It’s about when the new developer still doesn’t understand, the tests will complain before your customers can.
Profoundly Better Code Documentation
Most .NET programmers have seen structured XML used to document code. Likely, we’ve filled out several <summary>, <return> and <param> tags. Maybe several others. The intent behind them (and coding standards mandating them) is to insure our code is documented enough so the poor slobs who come after us know how to use it.
The problem here is a simple one — comments are pathological liars. They get out of sync with the actual behavior of our code. They often don’t reveal enough about the contract of the code. And when they do, they certainly can’t enforce what is documented (does that <param> accept null values? What happens if we pass a null there? What is the valid range for that integer <param>? What happens to that Stream when there is no data to write? )
Because comments aren’t executable, they have no way to enforce correct behavior. Unit Tests, on hte other hand, demonstrate exactly how to use a bit of code. Because they are executable, they simply cannot get out of sync with the code. Proper unit tests verify an exception is thrown when an invalid argument is supplied; that an integer is in range; what happens to a stream passed to a function.
Protection Against Breaking Changes
Let’s say you’ve been assigned a task that involves adding a feature. You find a class that does 90% of what you need it to do, so you make your changes. Your change doesn’t change the signature of a method, but it does modify the semantics of a parameter just a teeny, tiny bit. You build the code and sure enough, the compiler is happy. Of course the compiler is happy – you didn’t modify the calling syntax – merely the semantics.
So you run the tests and they break. You look at the breaking tests and find they were checking for that very semantic change. Turns out there are several assemblies that depend on the original behavior. If you had tested only your change, you’d never know you broke other code until the bug reports started pouring in. The unit tests caught that before your users could.
A unit test from a previous release demonstrates behavior that code out there somewhere depends on. It’s a warning you should heed.
Improves your Designs with Decreased Coupling
Wikipedia says:
A unit is the smallest testable part of an application. In procedural programming a unit may be an individual program, function, procedure, etc., while in object-oriented programming, the smallest unit is a method, which may belong to a base/super class, abstract class or derived/child class. — http://en.wikipedia.org/wiki/Unit_testing
Comp.software-eng.testing FAQ says:
A unit typically … does not include any called sub-components (for procedural languages) or communicating components in general.
Unit Testing: in unit testing called components (or communicating components) are replaced with stubs, simulators, or trusted components. Calling components are replaced with drivers or trusted super-components. The unit is tested in isolation.
Isolation is a big deal in unit testing. In almost every case, the only thing that should be tested in a unit test is one method of one class. All other participants in that method are stubs, simulators, and test-drivers. The only way you can get to that is by reducing your coupling. Unit tests are much easier to write if your classes depend on abstractions instead of concretions. You’ll also discover you simply cannot unit test a class that depends on the behavior of globals, singletons and most static members – no way, no how.
Test Driven Design (TDD) – not the same thing as unit testing, but a subset of ways to apply it — helps improve designs even further by forcing you to think about your code the way a consumer of your code would. As a result, our APIs are no longer defined by how something is done (leaky abstractions), instead they are defined by what our users want done, which then constrains our implementation (the TDD’s affect on unit testing is such a big subject it warrants its own post).
Better Tests
Finally, unit tests insure the quality of our code in a way no other tests can. All non-unit tests can only insure how a body of code behaves in ways the calling code knows how to call. Consider a function that counts words in a text stream. If that function is currently used only in a program that always supplies text streams of 2KB in size, and there are no unit tests for that function, then the only behavior we can verify for that function is that it can count words in text streams of no smaller or larger than 2KB of data, because the calling code can only handle streams of 2KB of data.
We simply don’t know what will happen if we get 0 bytes, 2047 bytes, 2049 bytes or 2MB of data.
Now, our requirements change, and we must consume just over 3K of data. Our functional tests start blowing up. We can either work through the call stack looking for the issue (let’s say it was an off by one error in our original function that used a 2KB buffer with a zero byte terminator … oops), or we could have had unit tests that leveraged the knowledge our word counting function used a buffer of exactly 2KB in size, so we could write edge case tests around that value).
In upcoming blogs, I’ll cover some ways to maximize the benefit of unit tests, while minimizing the pain.
1 Comment »
RSS feed for comments on this post. TrackBack URI
Leave a comment
Powered by WordPress with GimpStyle Theme design by Horacio Bella.
Entries and comments feeds.
Valid XHTML and CSS.
[...] you understand the primary beneficiary of unit tests are the developers, then it make sense we should make our unit tests as helpful to developers as we possibly [...]
Pingback by mentis vulgaris » Qualities of Good Unit Tests — March 2, 2010 #