Is there a danger in over testing?

John Paliotta

September 16, 2010

Gary at DevExpress asks the question: "Does Testing Make You a Bad Programmer?"

Overall, I agree with Gary's comments. But I disagree on the conclusion.

[via] Before the fashion for TDD and automated unit testing, programmers were adept at writing defensive code, just as my generation of cadets were adept at ducking when they saw a heavy rope coming. The newest generation of programmers must not allow the presence of unit tests to lull them into a false sense that their code operates in a “safe” environment. If that happens, if they rely only on tests and do not learn how to code defensively, then the answer to the question posed in the title of this blog post will be yes, testing does make you a bad programmer.

My friends and I have similar discussions almost every weekend at our kids' sporting events. Our society is constantly looking to remove the danger of failure from the lives of our kids -- “Everyone gets a trophy”; “We don’t keep score”, etc. By removing the stigma of failure, you never develop distaste for it; as a result, you are never driven to perform beyond your abilities, or encouraged to strive for excellence.

As someone with 30 years experience in the software development world, I would draw the analogy this way. Many software developers will do the least amount of testing allowed by their organizations. I frequently hear from customers that they already do low-level or unit testing. Often they say that it is "informal". Informal is code for “not mandated”, and as a result, “seldom done”.

Enter the "automated test tool". This is a manager's dream. My developers don’t want to test, my process does not make them test, but I want to improve quality. Hey, let’s dump our code into this tool and it will tell us if it is “good” or not. Automated testing is not the solution, intelligent testing is. Intelligent testing should ensure that the code meets its low-level requirements, and is robust (can handle edge cases). In my opinion, unit testing performs two significant functions:

  1. It leads to a more rigorous development environment. The process of building test cases will cause developers to give more thought to the boundary conditions and special cases that the code might encounter. Hey altitude is an integer, can the plane really fly at 2,147,483,647 feet?
  2. It provides a regression test facility that enhances your ability to re-factor code. One of the biggest reasons that applications get unwieldy over time is that not enough re-factoring gets done. We all have parts of our apps that developers are afraid to touch. Every time you change it something breaks. The reason is that the new logic that has been bolted on over time has no corresponding validation tests.

While unit testing is not the only solution, it is part of the solution. Unit testing combined with peer review of the results will lead to better quality. There is no doubt about that.