I’m a bit of a hypocrite when it comes to software testing. I see it as one of the most important indicators of software quality… but I sometimes just don’t want to be bothered with it.

Small teams often groan about how much work maintaining a testing system requires. They say it’ll slow down development velocity and will provide little benefit. And they have a point. Setting up a testing system and ensuring good test coverage is a non-trivial amount of work.

Typically, this untested state of affairs is arrived at when a small team or solo developer is working on an application, and they get to a release state by manually testing everything. And for something small, this might be sufficient. Then, another developer who doesn’t have enough background wants to work on it, or a quick iteration happens that breaks behavior. Then, the chorus of blaming about tests begins.

I suggest a truce between us all and that we set a few criteria for testing.

  1. When a project is small enough in scope that all important behavior can be tested quickly, you may not need tests. If you’re running a command line utility that only does one thing or a tool with only one use case, you might not need tests. Hobby projects may not need them, especially if failure is not catastrophic.
  2. Add tests the first time you get a little nervous to make changes. While this is entirely subjective, most folks know it when they feel it. Did you hesitate a little bit to commit that code? Are you confident that everything works, or only sorta confident? Time to add tests.
  3. Never add tests ‘just because’. Test for critical behavior and regression test every bug fix. It’s easy to get carried away with tests. But only some provide value. Test for critical behavior and bug regression. Add as needed, but don’t hesitate to remove or update what’s not working.
  4. Tests must always pass. No exceptions. If a test gives false positives or negatives, fix the test. If a test can fail without it being a problem, remove it. A failed test should mean ‘we cannot ship this’ so we never commit code that fails tests. No exceptions!

If we stick to these four things, I think I can be considered less of a hypocrite and perhaps a little more… pragmatic?