Specification: By Code, Tests and Contracts

Currently I’m taking further investments in thinking about the synergy of Design by Contract (DbC) and Test-Driven Development (TDD). This process is partially driven by my interests in Code Contracts in .NET 4.0 and other current developments at Microsoft (e.g. Pex). The basic idea I’ve got in mind is to combine DbC and TDD to take best advantage of both principles in order to improve software quality on the one side and to create an efficient development process on the other side. My thoughts on this topic are not too strong at the moment, so feel free to start a discussion and tell me what you think.

Before I come to discuss the synergy of DbC and TDD it’s absolutely necessary in my opinion to understand the characteristics, commonalities and differences of both concepts. This first blog post drives in this direction by looking at a very important aspect of both concepts: the formal and functional specification of code. It’s a starting point for further discussions on this topic (a 4–part comparison series follows). So let’s start…

Code-based specification

In essence, DbC and TDD are about specification of code elements, thus expressing the shape and behavior of components. Thereby they are extending the specification possibilities that are given by a programming language like C#. Of course such basic code-based specification is very important, since it allows you to express the overall behavior and constraints of your program. You write interfaces, classes with methods and properties, provide visibility and access constraints. Furthermore the programming language gives you possibilities like automatic boxing/unboxing, datatype safety, co-/contravariance and more, depending on the language that you use. Moreover the language compiler acts as safety net for you and ensures correct syntax.

Since such basic specification is necessary to define the code that should be executed, it has a lack of expressiveness regarding the intent of a developer and there is no way to verify correct semantics of a program with it. As an example interfaces define the basic contract in terms of method operations and data, but looking at C# a client of this interface does not see what the intent of a method is or which obligations he has to fulfill when calling a method or what state he can assume on return of a method. Furthermore interfaces can not guarantee uniform behavior across their implementations. TDD and DbC are there to overcome or at least decrease this lack of expressiveness at some points and to guarantee correct semantics.

Test-based specification

Let’s come to test-based specification using TDD (as well as BDD as „evolutional step“ of TDD). This is inevitably integrated in the TDD cycle. Every test written by a developer maps to a new feature he wants to implement. This kind of specification is functional and example-driven, since a developer defines by exemplary input/output pairs what output he expects as result of the test run under a certain input. With well-known techniques (stubs, mocks) he is able to run his tests in isolation, get reproducible test results and perform state and behavior verification.

Compared to code-based specification, test-based specification in a TDD manner is very valuable when it comes to expression of the behavior of a code element. It gives a set of tests that could at its extreme span the whole behavior of a code element. With their reproducibility the tests are indispensable when it comes to continuous integration to ensure correct behavior of modified code elements. Furthermore tests are a great source for other developers to show the intent of a developer for a method’s behavior and to give demonstration of the usage of a code element. There are other benefits and characteristics of TDD that will be discussed when comparing TDD and DbC altogether in subsequent blog posts.

An important aspect of test-based specification using TDD is that it’s done at a very granular level. For each new feature a test is written and if the present API doesn’t fit the needs, it will be extended or refactored. Thus TDD drives the API design as you go with your tests.

Contract-based specification

Another possibility for specifying code is contract-based specification in terms of DbC, thus defining preconditions and postconditions as method contract and invariants as class contract. With contracts you are able to define the basic obligations that a client must fulfill when calling a method as well as the benefits he gets in return. Furthermore invariants can be used to define basic constraints that ensure the consistency of a class. Thus with DbC a developer is able to define a formal contract for code that makes a clear statement of obligations and benefits in client/supplier communication (aka caller/callee). If a contract fails the behavior is clear as well: by failing fast the developer can be sure that there is a problem (bug) with his code and he has to fix it.

On a technical level there are several possibilities to define contracts. In Eiffel contracts are part of the programming language itself in contrast to Code Contracts that become part of the .NET framework. In any case contracts are directly bound to the code that they are specifying and express additional qualities of the code. Those qualities go beyond the code-based specification. In general contracts allow arbitrary boolean expressions what makes them a very powerful and flexible specification source. Nonetheless contracts only allow partial functional specification of a component. It can be very difficult or even impossible to define the complex behavior of a method (which methods is it calling, what business value does it have, what’s the concrete result for a concrete input, …) or to ensure certain qualities (e.g. infrastructure-related questions like „is an e-mail really sent?“) with contracts. Furthermore it’s impossible to use impure functions in contracts which would be necessary to express certain qualities of code (like expressing inverse functions: x = f^-1(f(x)), if f or f^-1 are impure). Test-based specification could be used here instead.

But let’s come back to the technical aspect: Contracts are wonderful animals in terms of extending the code-based specification by narrowing the definition of a code element and becoming part of the element’s signature. They can be used to define general conditions, e.g. physical constraints on parameters and return types. Thereby (for example with Code Contracts) the contracts can be inherited to sub-classes while respecting the Liskov substitution principle (LSP). Moreover contracts can be defined on interfaces and thus they are a valuable tool for expressing constraints and qualities of an interface that must be respected by every implementation. With that contracts are wonderfully strengthening the role and expressiveness of interfaces and complementing code-based specification at whole.

With the tight coupling to the code, contracts give immediate support for other developers how an API should be used. They express the developer’s intent (intention revealing) for his code elements, which leads to easier comprehension and usage of those components. They give some kind of checked documentation, which again greatly complements code-based specification and leads to fewer misusage of contracted components.

In contrast to test-based specification, contracts are employed at level of whole components by employing invariants to classes and pre-/postconditions to class methods. Moreover contracts lead to components with very few dependencies (promoting the SRP) since it’s difficult to write contracts for components with many responsibilities.

First conclusion

This blog post is intended to be a first preview of what I want to come up in the next time. It has given an overview of possibilities for specification of code elements with focus on test-based and contract-based specification in terms of TDD and DbC. It has shown some commonalities and qualities of both concepts. In conclusion TDD and DbC are both valuable in terms of the specification of code elements and in revealing the developer’s intent.

In my opinion it’s by no means a „one or the other“ choice. TDD and DbC act on their own terrains with their own benefits and drawbacks. There are overlaps, but in general they can naturally complement each other. It’s time to think about ways to leverage a conjunction of both concepts, isn’t it?

[To be continued…]