Thursday, May 6, 2010

Why invest in TDD? Refactoring.

I've been asked lately why test-driven development so important. Apparently many teams (or their management) still think of methodical unit testing as an additional luxury expense. "We don't have time for tests", "Test code can't be sold" are common arguments.

Without digging into testing methodology, design considerations and consequences, I'd like to focus on benefits in the broader context of development process. Most modern methods - especially agile - aim to address the problem of code decay. During a software project's life-cycle the code quality deteriorates, and the cost of introducing new features increases significantly. Refactoring is the main means to improve code quality and protect against decay in order to keep development and maintenance costs manageable.

Note: this diagram is an illustration
We can come to the conclusion that continous refactoring should be an organic part of the development process. The main challange is though, non-trivial refactorings are potentially unsafe. Even a carefully done refactoring has potential to breaking functionality. Most developers - rightly - don't do unsafe operations often (otherwise they would sit in front of a debugger all day long without being productive). One of the most important goals of testing is to make refactoring safe: make resulting problems instantly apparent and easy to pinpoint. According to my experience they serve this purpose very well.

I highly recommend the following book for reading: Refactoring - Improving the Design of Existing Code from Martin Fowler. He discusses this topic in great depth.

7 comments:

  1. Ugh! Ask yourself why the code deteriorate, and fix that problem instead of fixing the symptoms.

    Modern methods my ass - Fowler is only experienced in crud business applications and object orientated language.

    People had written software to navigate planes, play chess and put men on the moon without your agile crap.

    I find most agilist have myopic view on software.

    Do you even have numbers to backup your graph? Or you just decide to draw it up based upon your instinct like kent beck?

    ReplyDelete
  2. Hi Anonymous,

    Code deteriorates because there is a lot of uncertainty involved in almost all development projects. To put it more simply: you'll very often require to do previously not expected changes.

    You loose efficiency if you let your code degrade when doing changes, so you need to prevent that. Refactoring by definition is the way to achieve that. Testing is there to make refactoring more feasible.

    In what way would this be a "symptom fix"? Is there a fault in the above logic? Why pointing out that all development tasks are doable without all this when we're discussing doing things efficiently? All tasks are doable by writing binary code if efficiency is not a concern.

    ReplyDelete
  3. This is much interesting and I have to admit that (as you have written on your last comment) "you will very often required to do previously not expected changes" is simply the truth.

    But as "Anonymous" pointed out, what are the numbers which you used to build the graph? Also, for developers, it is quite easy to understand the truth about "code rotten", but do you have any more arguments to convince management people?

    ReplyDelete
  4. Chromano:

    The graph is not my creation, I found it on the web somewhere. I think even if it were based on real numbers, it wouldn't have much meaning without providing definitions of "feature", "release", an accepted metric of "refactored system" and so on. I used it to illustrate our goal here. In my experience though it can't be too far away from the truth.

    Your other question is very interesting and also very tricky. Business is not interested in code quality, but interested in things like time, scope, cost, etc. As a developer you know the relation between code decay and cost. Things are not so unaligned, just needs to be communicated properly. It can help a lot to have a commonly understood metaphor for code decay.

    There is very suitable one created by Ward Cunningham called the technical debt. When you implement a feature in a "dirty way" (the code deteriorates) you create a debt. When you implement new features you need to pay the interests of your debts (more difficult to work with bad code). At any point you have an option to repay your dept (refactor to restore code quality) and become free of paying any interests (work efficiently with good code). The total cost is minimal if you don't create debts (continous refactoring).

    Just some quick ideas.. this topic is huge in itself, can't be dealt with so simply. I may try to take a stab at it in a later post.

    I'm looking forward to receiving further comments on this topic.

    ReplyDelete
  5. Hi,

    I find this "technical debt" concept very useful and in such way this can be even presented to a manager.

    However, it turns out that we can have two different situations. The first one, is just like Ward Cunninghan has described, i.e, you write "dirty code" because you find yourself in a rush, everyday.

    In the second case, you have the time needed to write good code, and you do it... but then new requirements appears (a normal situation) and they specify new features that you didn't prepared the code for. You know, it is impossible to be generic enough to cover all possibilities, then I think continuously refactory would be perfectable adaptable to this situation too.

    That's it -- not exactly a question, but I would like to share with you my thoughts and see what do you think about it :)

    ReplyDelete
  6. TDD is unrealistic. You spend time writing wrappers around many 3rd party tools that it becomes pointless.

    I understand the concept of TDD, and for the most I think it's extremely accurate, but how do you write TDD for distributed file systems? How do you write TDD for distributed / parallel tasks?

    These things become harder to think about and take up almost 1/2 of the problem of actually writing the code to do the tasks.

    I find the idea Utopian, but, for anything complex that requires multiple environmental factors, TDD becomes more of a burden to keep everything under thin-wrappers and investing in mock designs than productive.

    Methodologies are a good general guideline, but shouldn't be zealously preached -- especially with a graph like that.

    ReplyDelete
  7. I don't think it's unrealistic or even utopian. There are a bunch of projects out there running happily with TDD and many others that could benefit from it.

    I agree with you that this is a guideline and doesn't necessarely apply to all projects. It is common sense that any methodology can be abused.

    ReplyDelete