Why is it important to write automated tests? I’m asked this question far too often by software engineering teams. The usual response I receive after asking someone why they don’t write tests usually is because “I don’t have time”. Nothing could be further from the truth. In reality, valuable time is saved by writing tests first. Writing tests is an important and fundamental concept to our craft. So much so that I thought best to write about it.

But as with all things, first a bit of context.

The Agile Development Methodology

Agile is a software development methodology that borrows from Scrum project management practices, Extreme Programming (XP), and Test Driven Development (TDD).

It’s not my intention to write about Scrum, XP, and TDD in excruciating detail since there are many excellent books on these subjects. I would, however, like to share my view on why TDD is foundational to any software engineering effort.

Scrum

As most are aware, Scrum is an iterative, incremental framework for managing complex work and though intended for management of software development projects, it can be used as a general program management approach. Contrary to popular belief, however, Scrum is not an acronym. First used to describe hyper-productive development in 1987 by Ikujiro Nonaka and Hirotaka Takeuchi, Scrum actually refers to the mechanism used in rugby for getting an out-of-play ball back into play.

Many software engineering teams still practice the Waterfall development methodology and for some projects – in particular when requirements are static and rarely change – it’s an appropriate practice. The building of a jet propulsion engine is a good example of a project that likely benefits from the Waterfall development methodology. In contrast, Scrum’s two pillars are team empowerment and adaptability:

1. Team empowerment: Once teams are given work to do, they are responsible for figuring out how to do it. The team does the best it can during each iteration. While a team works, their only interaction with management is to tell management what is getting in their way and needs to be removed to improve their productivity.

2. Adaptability: Scrum uses “punctuated equilibrium”. The team maintains an equilibrium during each iteration, insulated from outside disturbance. Iteration are generally punctuated every thirty days so that the team and management can evaluate what should be done during the next iteration; this decision is based on what the team has accomplished and what the environment dictates is the next most important thing to do.

Velocity

It’s difficult to discuss the importance of writing tests without first discussing the notion of velocity and while I could write a dissertation just on the notion of velocity, it is simply defined as how many story points of effort a team can complete in a single iteration. Once established, velocity can be used to plan projects and forecast release and product completion dates. A team’s velocity is an important key performance indicator.

eXtreme Programming

From XP, we borrow pair programming and continuous integration. Code reviews are good so let’s do them all the time; this is pair programming. Measuring code quality and ensuring a source level change didn’t cause any regressions is good so let’s do it as often as possible; continuous integration.

It should be no surprise that writing automated tests and measuring code coverage go hand-in-hand with continuous integration.

Test-Driven Development

For any particular software engineering effort, there are three axes to control: quality, schedule, and scope. Quality remains locked at “high” and so the engineering team must choose how to balance the remaining two.

Clean code that works…now. This is the seeming contradiction that lies behind much of the pain of programming. Test driven development replies to this contradiction with a paradox – test the program before you write it.

• Test Driven Development: By Example, 2002

Regardless of your development methodology, writing tests are important. Automated testing underpins many Agile practices, most significantly as part of Continuous Integration. TDD is a process by which the tests are written before the code itself. Through a rapid cycle of adding new tests, making them pass, and then refactoring to clean code, the software design evolves through the tests.

Red, Green, Refactor

When a test is first written, it most likely won’t even compile - the implementation hasn’t been written. Modern IDEs allow a developer to rapidly create code and then refactor it into shape. After adding a test, a developer strives to reach a “green bar” (all tests passing) as quickly as possible, and then refactors backed up by the safety of the tests. This quickly becomes a cycle of “red, green, refactor”, adding a kind of rhythm to the development process. When a new feature is required, tests are added, code is written to make the tests pass, and finally refactoring removes any duplication.

Benefits of Writing Tests

Whether your project is open source, commercial, or corporate, there are clear and important advantages to writing automated tests.

1. Courage. Writing tests allows one to move from a state of uncertainty to that of courage. When you write tests, you know the state of the your code. Intimately. You know if your code works or if it doesn’t. You know if your old code, and the code of your colleagues works as expected.
2. Communication. Tests can also be used for communication. This works well to communicate what the software does. We all may not know how to speak the same language but as a member of a particular software engineering team, we all know how to communicate using a specific programming language. We all can read and write code with good clarity.
3. Documentation. Beyond shore-to-shore communication, tests also make a good candidate to use as examples in documentation and in comments.
4. Refactoring. Setting your minimum allowable test coverage percentage to a respectable number (90 percent and greater) allows you to refactor your source code with confidence. Use cases pivot. You refactor the code. Then simply make all failing tests pass again.
5. Design. Writing tests first leads to a better design.
   1. It forces you to think about the shape of your API up front; to really understand how your consumer uses the API before you spend valuable time writing the implementation.
   2. It allows you to understand any possible frustrations and complexities with the shape of your API before your consumer.
   3. It saves time by ensuring the API satisfies your design goals and nothing more.

All of these points are critical to a team’s velocity.

Rules of Engagement

The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY and OPTIONAL, when they appear in this document, are to be interpreted as described in RFC-2119.

In the end, automated tests are just code. Even with best intentions, they can be written incorrectly and contain bugs. Here are a few rules to consider.

1. Keep unit tests small and fast. Ideally the entire test suite SHOULD be executed before every code check in. Keeping the tests fast reduce the development turnaround time.
2. Tests MUST be fully automated and non-interactive.
3. Keep tests independent. To ensure testing robustness and simplify maintenance, tests SHALL NOT rely on other tests and SHALL NOT depend on the order in which other tests are executed.
4. A test MUST leave the environment exactly as it was before execution began. This is an extremely important point since test execution order is never guaranteed and so environmental changes may cause sporadic test failures, which can waste precious time.
5. A test MUST contain at least one assertion.
6. A test MUST contain 3 logical sections: arrange, act, and assert.
7. A test MUST cover boundary cases.
8. A test fixture MUST provide negative tests. Negative tests intentionally misuse the code and verify robustness and appropriate error handling.
9. Write tests to reproduce bugs. When a bug is reported, write a test to reproduce the bug (i.e. a failing test) and use this test as success criteria when fixing the code.
10. Conduct peer code reviews on all tests.

Regarding bullet 6, William C. Wake wrote about using the “arrange, act, assert” pattern in 2003. Here’s a summary:

• Arrange: What is being set up and initialized is contained in the arrange section.
• Act: What method is being executed is contained in the act section
• Assert: What determines the outcome of the test is in the the assert section.

I’ll write about dummies, mocks, stubs, and fakes in a future post.

Conclusion

With test-driven development, robust code is delivered along with a comprehensive suite of tests, allowing developers to make future changes with confidence. I firmly believe that unless you’ve been a contributing member of a software team responsible for a complex code base that it may be difficult to appreciate the critical importance of having a suite of automated unit, integration, and story tests.

Move away from a position of uncertainty to one of confidence. Write tests.