As you certainly already know, there has been a lot of discussion about Microsoft’s understanding of Test Driven Development (TDD) in the last few days. The entire discussion has been interesting to follow and in case you missed it, here are a few postings about the topic to get you started:
- Microsoft Guidelines for TDD Wrong
- Microsoft fails miserably to explain or promote Test Driven Development in Team System
This discussion motivated me to post my opinion about TDD. To put it short, I don’t believe in the TDD methodology. Don’t get me wrong, this doesn’t mean I don’t believe in testing in general. I think testing is very important and definitely won’t consider shipping a product which doesn’t have an extensive test suite (manual or automated tests), but I am not necessarily a big fan of the TDD process. For me, the entire TDD process is too unproductive. I prefer concentrating on one task for a while rather than switching between testing and development all the time. Furthermore, I don’t think writing the tests before having the actual code is the optimal solution. In my opinion, writing really solid and good tests involves knowing the implementation of the code to test. How can one reach a good code coverage when the inner workings of the system are unknown at the moment of test writing?
6 Comments
>> How can one reach a good code coverage when the inner workings of the system are unknown at the moment of test writing?
That’s why you have automated code coverage tools, like NCover.
A single task, as you mentioned, in TDD is writing a test and the code to make it pass. There’s nothing about it that makes you jump around.
Hi Miki,
Thanks for your comment. I know about code coverage tools. The problem I see is nonetheless still there. In TDD you first write the tests and then the actual code. If you then measure the code coverage of your existing tests, it’s very likely that you need to adjust or rewrite them. So why not delay the test writing until you’ve finished your code in the first place? Saves time in my opinion.
And regarding the jumping, as I understood TDD, it’s a continuous switching or jumping between writing tests, writing production code and refactoring. The explanation I refer to is posted here, for example:
http://samgentile.com/blog/archive/2005/11/22/32116.aspx
I think switching between these tasks has pros and cons. One of the cons is that concentrating on one of these three tasks alone is by far more efficient for me then doing all of them quasi simultaneously. Constantly doing context switches between these tasks (especially between writing tests and writing production code) takes time which can be spent better in my opinion.
I understand and admit that TDD has a lot of pros, but the "old" approach of doing a rough design first, then writing and/or refactoring the actual code and then in the end writing the tests is still working better for me.
Regards,
Tobias
>> The problem I see is nonetheless still there. In TDD you first write the tests and then the actual code. If you then measure the code coverage of your existing tests, it’s very likely that you need to adjust or rewrite them. So why not delay the test writing until you’ve finished your code in the first place? Saves time in my opinion.
The point about writing tests first, is that you can design in advance and evolve that design.
Writing a test enables you to write the interface the way you want it, so you don’t have to discover that some function requires some object, and creating that object requires you to jump through hoops.
Also, Writing the test first and mocking other classes in that test, enables you to write tests for code that hasn’t been written yet.
For example, see my post:
http://orb-software.com/cs/blogs/green_dragons_lair/archive/2005/11/21/157.aspx
where i talk about working with the Session variable in ASP.Net in a unit test. I’m not showing the tests there, but it was developed using TDD.
I’ve often been surprised by the code that I’ve written with TDD, how flexible it becomes.
>> And regarding the jumping, as I understood TDD, it’s a continuous switching or jumping between writing tests, writing production code and refactoring. The explanation I refer to is posted here, for example:
http://samgentile.com/blog/archive/2005/11/22/32116.aspx
>> I think switching between these tasks has pros and cons. One of the cons is that concentrating on one of these three tasks alone is by far more efficient for me then doing all of them quasi simultaneously. Constantly doing context switches between these tasks (especially between writing tests and writing production code) takes time which can be spent better in my opinion.
I don’t see writing tests and production code as such a seperate things, it’s like writing an interface and then writing the implementation.
Let me give you an example:
- I want to have a function that does something
- I write a test that calls DoSomething(), and call the test DoSomethingPasses, and assert that the return value is zero.
- The function doesn’t exist, so I create it and throw a NotImplemented exception. Run the test, it fails. This gives me validation that the test can tell me when it’s wrong on the base level.
- Next, i add the production code, just enough to make it pass. So if I’m expecting it to return zero for success, i’ll just return zero. Run the test, the test passes.
- Next, I write another test, this time DoSomethingFails, and assert that the return value is not zero. Run the test, it fails because the return value was zero.
- I modify the function to check some values and return the proper value. Run the test, it passes.
And so on. Note that I have been focused on the same task all the time, even if I changed the files I’m working with, but then, that’s pretty much the same with non-TDD development.
Of course, there are refactoring stages between each test that I haven’t covered.
Another very important benefit of TDD, is that you end up having a suite of tests, that can immediatly tell you when a change that you made broke something.
>> I understand and admit that TDD has a lot of pros, but the "old" approach of doing a rough design first, then writing and/or refactoring the actual code and then in the end writing the tests is still working better for me.
Yes, but you’re missing the benefits of TDD then, such as evolving the code.
I’m not a TDD zealot, I pick and choose what I like from it, but from my experience, TDD/Agile has made me a lot more productive and gives me assurance when making changes that I’m not breaking stuff. When I go back to older non-TDD projects, I’m afraid of making changes, and testing those changes are very hard and doesn’t really give you a 100% assurance that they’re working.
Hi Miki,
Just to clarify up things, I too believe in automated tests. For the SmartInspect libraries, we recently reached 3000 tests. These tests have nearly twice the amount of lines as the actual production code. For every feature we add we end up writing lots of tests. The only difference to TDD is that we mostly write the tests after the design and implementation are almost done and not before.
During development I often change my mind about implementation details and refactor the code until I like it. Now, if I would write the tests before writing the actual code, I would either end up adjusting/rewriting the tests or having bad code coverage results. Additionally, I’m sure that TDD works for simple tests like in the example you gave, but normally tests are more difficult than checking the result of a function call. I think of integration tests, for example. With integration tests I mean tests which affect more than one simple unit, say a set of classes or even an entire library. Integration tests can get much more difficult, simply because they cover a lot of production code.
As an example, for the SmartInspect libraries I wrote a custom TCP Server just for having an adequate tool to test the TCP handling of the SmartInspect libraries. With this server, we have been able to write automated and robust tests for verifying the connect timeout, for verifying a correct DNS lookup and so on. As you might imagine, these tests aren’t that simple and depend on a lot of factors. In my opinion, it would be highly impracticable or even impossible to write such kind of tests when the design isn’t done.
Regards,
Tobias
I don’t disagree with you, integration tests are quite a different beast, I just think that TDD/unit tests helps you design better, but from your description I can see that you have the design phase quite under control
Hi Miki,
For me, integration testing and unit testing are nearly the same, except that most integration tests are more complex than simple unit tests. Both kinds of tests should ideally be automated and only differ in complexity and lines of code in my experience.
Regarding the design phase, normally we are doing a rough big-picture design first and then break it down into several smaller pieces. For these smaller pieces, TDD might indeed be an alternative way of doing the development. But for the big-picture design, I still think that the original way has more benefits than TDD.
But in the end, it’s irrelevant anyway whether you use TDD or another approach. The only thing that matters is to have a reliable and robust regression test suite at all
Regards,
Tobias