TDD is Kanban for Code

The other night Cynthia and I were having drinks in the Tower Bar of the Hotel Hafen in Hamburg (highly recommend for the view if not the service) with Henning Wolf and Arne Roock of it-agile when I casually mentioned that test-driven development was kanban for code. Arne teaches kanban but the connection wasn’t obvious to him, so I sketched my idea (see napkin above). He seemed to understand (he kept nodding, anyway), but I thought it prudent to follow up with a post to make sure I’d thought the whole thing through. Arne, this one’s for you.

Kanban

The goal of kanban is to increase the value of a production process. Kanban increases the feedback in production by limiting the amount of work in progress at any one time. Without the “safety” of inventory, troublesome process steps and connections between steps have nowhere to hide. With better feedback, the steps can be optimized (or eliminated) and their connections improved. The result is higher throughput and lower latency and variance.

Kanban works by only producing on the basis of demonstrated need. While one finished product is being assembled, upstream steps are producing the parts for the next finished product, timed to arrive just as they are about to be used. When a kanban system is humming, it produces a steady stream of finished products with a short gap between the receipt of an order and its fulfillment. When it isn’t humming, the source of the dischord is likely to be clear.

Test-driven Development

Test-driven development (TDD) is an alternative programming workflow where features are divided into a series of automated tests. The tests are chosen so that satisfying the spirit of all the tests implies that the feature as a whole works correctly. The cycle goes like this:

  1. Write the next test (while several tests may be outlined, only one is written at a time)
  2. In pessimistic languages, add enough stubs so the test compiles. It should fail when run.
  3. Refactor, if necessary, to prepare for the implementation.
  4. Change the logic of the system so the test passes.
  5. Refactor to eliminate duplication and other excess complexity

Repeat until no more of the tests implied by the feature would fail. Then the feature is complete.

The Analogy

If TDD is kanban for code:

  • What is the product?
  • What is the kanban card?
  • What are the production steps?
  • How does demand flow backward and products flow forward?
  • What are the feedback loops?

The product of software development is two-fold:

  • The behavior of the system
  • The options for extending and modifying that behavior in the future

As difficult as it can be to precisely specify and validate the behavior of the system, it is even harder to measure a system’s option value. Finding the right balance between these two goals is one of the big challenges of development, especially as different phases of the business cycle require different proportions. I’ll describe in a moment how TDD addresses this balance, both in theory and in practice.

The tests are the kanban cards in this system. Each one is a request for a change of behavior. “Please change the system so if the customer is a smoker the premium is 12% higher”. This will require changes to the logic of the system. It may also require changes to the structure of the system so the change to the logic is easier to make correctly and efficiently.

The post-success refactoring commonly emphasized in descriptions of TDD isn’t explicitly called for by the tests. It is work done for future benefit. If the future happens to be the next test and implementing it is easy because of cleanup done after the last test was satisfied, then the payoff is immediate. From the kanban perspective, though, post-success refactoring is over-production, the worst of the seven wastes of the Toyota Production System. Still, most TDDers, myself included, clean up after a test is passing as a matter of habit.

The behavior and options are the product, the test is the kanban card, so that makes the production steps the changes to the logic and the changes to the structure of the system. In TDD, these changes are not begun until a test is failing. The changes flow forward into the product, as demonstrated by the passing test, and the lack of collateral damage is demonstrated when all previous tests also pass.

Feedback loops fuel kanban. In TDD, the programmer gets feedback in seconds about whether the logic implied by the test is the logic he writes. He also gets feedback about whether the design supports the logic changes based on how hard or easy those changes are to get right.

Conclusion

This isn’t the whole story of development. Post-success refactoring doesn’t fit into this picture but is a common practice. Is this just because of our still-evolving understanding of TDD, or is it more like the preventative maintenance needed to keep kanban systems running smoothly? Should some or all of it be deferred until immediately before it is needed? What about other valuable software development activities like automation, tool building, and exploration?

We could also pull back and look at development at a larger scale and see the features as kanban cards, each one “pulling” a collection of tests into the system, triggering coding. From even further back, a process like Customer Development can be seen as “pulling” demand for features.

In any case, TDD is kanban for code:

  • Product = behavior and options
  • Kanban card = test
  • Production step = coding and refactoring
  • Feedback = effort and test results

Arne, how does that work for you?

27 Comments

Steven A. LoweNovember 9th, 2010 at 10:33 am

The post-success refactoring is not waste, it is kaizen – incremental improvement. It is also called for by the reduce/reuse/recycle mantra.

Tore GreenNovember 9th, 2010 at 11:01 am

@Steven: I think Kaizen normally refers to continuous improvement of the process, not the product?

I agree with Kenn that doing any work after the test has passed breaks the otherwise very good kanban analogy.
It is overproduction as you are doing work after the product was “shippable”. On the other hand, it is a good investment as you most likely would spend more time if you defer the same refactoring to a later time — doing it right after implementing the behavior means you have knowledge (about the implementation and the needed refactoring) that you will forget and have to re-acquire if you wait.

So this is a case on mental work maybe being slightly different from physical production work. Who would have thought?

Arne RoockNovember 9th, 2010 at 11:25 am

Kent, great post, and thanks for mentioning me! Sounds good to me (at least the part that I understand. Remember me as a very old school C64 geek?)
I was thinking of the kanban card as well. Perhaps it’s not the test itself but the change from green to red or from red to green that is the signal to pull a new test or refactoring (though there’s no change of colors when refactoring is done, right?)
Arne

Jens ColdeweyNovember 9th, 2010 at 11:36 am

If options are the product, refactoring is part of the product: it opens future options. I see only waste if you don’t refactor. Anyhow, this is a great post, thank you, Kent!

Paul NelsonNovember 9th, 2010 at 12:00 pm

I am frequently asked if my approach to TDD – which includes writing “throw-away” code quite frequently – is a valid approach. I think it is because the throw-away code (I also call it cheating: the lack of sufficient test coverage allows me to get away with an incomplete implementation) is also a kanban (signaling card) to add the next test.

So maybe TDD is bi-directional kanban? A test is a signal to add the next piece of code necessary, and the fact that I can get away with a sub-optimal implementation is a signal to add the next failing test.

Of course this begs the question “when viewing the process in the opposite direction (code signaling the “pull” of a test) what is the product?”

NatNovember 9th, 2010 at 12:05 pm

@Tore: source code is not the product, a running system is. Refactoring improves a team’s ability to produce that. So refactoring does, to me, appear to be kaizen.

Bruno ThomasNovember 9th, 2010 at 12:05 pm

I agree with Steven, refactoring could be seen as Kaizen (little improvements of the overall build process), or 5S (cf http://en.wikipedia.org/wiki/5S_%28methodology%29 )

Arne RoockNovember 9th, 2010 at 12:09 pm

Two more thoughts on this:
1) I don’t consider doing work after the tests as waste. It is highly valuable for the customer because it keeps the code maintainable and it enables options for future changes.
2) This example shows that the steps visualized on the ksnban board are not persons or roles but process steps. With TDD usually all steps are performed by one person or a pair of two.

Ilker CetinkayaNovember 9th, 2010 at 12:09 pm

A very good and matching analogy. As for the post-success refactoring, I tend to see it as mandatory for the product itself. From my perspective, it’s no overproduction at all.

First, the definition the product includes options and not only behavior. Refactoring clearly pays into options here. Second, the “mechanics” of post-success refactoring is comparable to “getting (back) to a reasonably suitable position” for the next production step. One might think of a robot in a production line as a metaphor for this “prep for next” aspect.

As for your closing questions, I believe we need to continue inspecting and researching on the act of software creation. As Tore mentioned already, mental and physical work are different things. This is a fact we need to accept and respect when drawing analogies (or generalizing processes). Especially when it comes to automation, tooling and exploration techniques. We might address such activities to factors of productivity improvement and throughput maximization. However, it seems to me to be more of a background (secondary) consideration for the “TDD is Kanban for Code” statement.

A very inspiring view to TDD as well as Kanban which will keep me thinking about it for quite a while, for sure.

Vicente BoschNovember 9th, 2010 at 1:08 pm

@Paul: Would the product on the inverse flow be a complete specification ?

Why can the production of a car to its very detail (not just the car but the entire flow of production) be defined at the start ( even with a prior estimation of possible production error) and we need to define the specification in cycles with production ?

I think that maybe the reason is because the code is in itself the most complete of specifications but aiming for it would be producing in itself the product ( due to the conceptual nature of software).

adminNovember 9th, 2010 at 3:15 pm

Jens,

I agree that options are part of the product, but they seem in this story to exist outside the control of the kanban cycle. Where are the cards that say, “Cheaper to deploy” or “easier to optimize” or “easier to internationalize”? It could be that this is where the analogy breaks down, that there is no such thing as a quanta of flexibility/readability/reliability.

JooyunghanNovember 9th, 2010 at 3:23 pm

Code smells can be seen as defects. From the perspective of “own-process completion(自工程完結)” refactoring is a required step in the process.

Steven GordonNovember 9th, 2010 at 3:39 pm

Not an expert in Kanban, but my impression is that the tasks that would sum up to a completed product are planned upfront and represented on the task board before starting work on them. On the other hand, when doing TDD, we do not plan out all the tests and name them before we start making the first test pass.

This would seem to be an important way in which the analogy fails to hold.

adminNovember 9th, 2010 at 3:46 pm

I do. I start a coding session with a list of the test that I’d like to pass, adding new tests as they appear to be necessary.

Steven GordonNovember 9th, 2010 at 3:52 pm

With respect to refactoring, I have found that it works for me to:

1. After a test has just passed, only do the absolute minimal amount of refactoring necessary to make the code expressive and eliminate obvious duplication.

2. After writing the next test and before making that test pass, only then do any intense refactoring and then only to patterns that would facilitate how I want to make this next test pass.

I do this to avoid YAGNI when refactoring. YAGNI would seem to be the code-equivalent of waste.

With this approach, the amount of refactoring while the software is “test-complete” is very minimal indeed.

Steven GordonNovember 9th, 2010 at 3:55 pm

> I do. I start a coding session with a list of the test that I’d like to pass, adding new tests as they appear to be necessary.

Is this practice common among TDD experts?

MIchael HönnigNovember 9th, 2010 at 6:34 pm

IF there is a requirement for keeping options, shouldn’t there be a test for “code is comprehensible to allow further changes“? Such a test could be based on metrics like McCabe. And thus, we would still see a red bar until refactoring is done.

Ok, there might be no such coplete collection of metrics, but that only means that we haven’t yet automated this particular test and not that the requirement “clean code“ does not exist.

Jusr my 2ct
… Michael

MortenNovember 10th, 2010 at 1:09 am

I’d like to see the post-success refactoring as a part of production. If you leave your tools around or clutter the workarea it will slow the next cycle of development down.

It’s like cooking, you could say that your done cooking as soon as there is an eatable delivery. But unless you continuesly clean up your workarea there will be a pile of work that still needs doing. I.e. the cleaning up. (not to mention the hyigen factors that come from keeping a clean workbench while cooking)… Now I’m hungry.

St. StrittmatterNovember 10th, 2010 at 3:09 am

Thanks for this article. I love TDD and now I see how close it is to Kanban!

Michael DubakovNovember 10th, 2010 at 3:53 am

Make conclusions based on analogy is often a bad idea. To mee TDD != Kanban. Sure, both have feedback loop, so both have common steps/properties of a cyclic process. But that is it. Kanban Card != Test.

Feedback loop in TDD is seconds. Feedback loop in Kanban maybe days, maybe weeks, maybe even months depending on many factors. It is a HUGE difference, which makes many TDD practices useless for Kanban. For example, refactoring is a cleanup without functional changes. What is refactoring in Kanban? Almost any card on Kanban board changes functionality somehow.

So I don’t think this analogy is viable neither useful.

adminNovember 10th, 2010 at 6:21 am

Michael,

Thank you for the perspective. I have seen kanban systems in action where the feedback loop was in seconds (and almost got run over by a forklift in the process).

One of the lessons of kanban is that you uncover problems by reducing the work in progress inventory. At our recent class we had exercises where we deliberately wrote larger and smaller tests as a way of understanding the consequences of varying WIP. By the end of the class the students refused to write larger tests, which I found both funny and informative. However, the analogy was helpful to me in identifying this issue.

That said, if the analogy doesn’t help you, there is nothing wrong with that. Do you have models for TDD that do help you think about it?

Paul NelsonNovember 10th, 2010 at 8:18 am

Vicente,

The idea of a complete specification being the product of viewing things in reverse sounds right, and yet paradoxical: the specification predates the implementation, so how could it be the product? The word “complete” is the key. And it is common for TDD to push changes back into the specifications through conversations and collaboration with the Customer/Stakeholder/Business Sponsor. These are triggered by better visibility into the problem space that comes from working on the implementation.

Paul NelsonNovember 10th, 2010 at 8:26 am

Hi Steven,

I suspect that the practice of writing down a list of intended tests is common among TDD experts. I frequently start with a list of tests, but not always. I can say, however, that the list I come up with ahead of time has never ended up being complete. Driving to a good implementation through tests always uncovers deficiencies in my initial mental model, causing me to add tests to validate the revised model.

Federico ZuppaNovember 10th, 2010 at 11:21 am

Hi Ken. I love the analogy. It seems to me that TDD is a pull-system. In your last comment, you said you “wrote larger and smaller tests as a way of understanding the consequences of varying WIP” and I was thinking that: 1) a shorter test not only minimizes the WIP, but also improves the lead time (being the lead time the time since you write the failing test until that test is passing & the code cleaned up). 2) TDD is the most natural way of minimizing the WIP, as you are working on one test at a time. Working in more than one (or widening the TDD step) increases the WIP, affecting the flow (which is something that naturally happens programming not test driven). I was also thinking that the analogy works well, in terms of visualization. Kanban tries to visualize the process and TDD (having a list of passing/not passing tests) allows you to visualize very clearly your progress related to a feature.

Steve ZhangNovember 13th, 2010 at 3:27 pm

I agree with the idea that the post-success refactoring is a must, not a waste, followings are my thoughts:
1. refactoring is Kaizen, Steven already mentioned.
2. refactoring pays off the technical debt, if you don’t do it right now, then next time when you add the new behaviour to the system, you still have to do it, because we have to adjust the system structure to adapt the new stuff;
3. I think post-success refactoring is also related to “Build Quality In”, which improves the internal quality. By refactoring, code quality is improved: maintainability, readability, flexibility, and will improve developer’s productivity, because cleaner code will help to prevent potential defect being generated.

adminNovember 14th, 2010 at 8:04 am

I agree that refactoring is a must, but how much and when? What TDD as kanban suggests to me is that I should try moving refactoring to just before making the test pass and see what happens.

Steve HayesDecember 30th, 2010 at 8:02 am

Late to the party, but I’m about to write about this post at work. I’m rather ignorant of Kanban, but in a factory setting wouldn’t there be a step where you cleared up the waste from production – offcuts, oil on the floor etc? It seems to me that the post-implementation refactoring is similar to this. In principle you and stop and ship with a mess in your factory, but you don’t (most of the time). Same with code.