Unit Testing - Watch out!

I just thought that I'd write a little article to discuss some of the things to watch out for when unit testing. People tend to talk more about why it is or isn't good, and that you should use it, I'm proposing to tell you what to do when you do!

The first thing, the level of confidence in the test of the application is only as good as the tests that you write. Seems obvious, right? Using unit testing techniques does not make your code any more manageable, or correct if the tests that you write are of a poor quality. so all that you can say about unit tests, is that the code passes for the tests, and never that it 100% works, unless of course you are 100% certain that you have catered for all eventualities in the code.

Using code coverage as a tool to target your tests will not guarantee anything about the quality of confidence you should have in those unit tests. Here, I mean that having 100% coverage means that all the code is tested; it doesn't. What it means is that all the code has been exercise, but not that the code has been exercised with every possible value. We tend to use boundary conditions and the like to use in our tests reasoning that these values will best test our code. They don't, they're only representations.
Code coverage, you might not be able to exercise 100% of the code, if your code uses some .NET object that you can't exercise all the CLI for. You should find out whether that is the case though. It can often be an indication that the boundary condition by itself is woefully inadequate.

Integration tests. Until you do integration tests, you can't only speculate as to how well the code will work in the real environment. What happens if you get data that you didn't expect? What it means is that you write more tests, so that it doesn't happen again, even if the data was in error. The further upstream you go in terms of integration tests, toward a full system test, the better the tests are, and the more likely it is to fail. You're using real data, in real situations, and so can have a certain amount of confidence that the components at least work in that real world scenario. Look at exercising as much of the code using the coverage tools again, obviously some parts, such as testing exception handling belong only in the unit tests, but anything that can have data driven through it, to get as much coverage as possible is good.

If we do things right, then when we find errors, we write more tests, so that the tests become more and more complete, and our experience in where the tests could fail with the components increases, and thus gives us a better indication and knowledge of the risks involved with deploying the software.

So long as we bear in mind that we're never going to be able to say with 100% certainty, that a component is 100% bug free, we're probably safe. What we do with unit testing techniques is mitigate the risk, improve our knowledge of the system and its inner workings thus limiting risk and document part of the system.

These techniques are very useful, if used to their full potential, but never make a sweeping statement that the software is 100% bug free because all the tests pass, since that proves nothing.

Happy coding!