Don’t Cross the Beams: Avoiding Interference Between Horizontal and Vertical Refactorings

As many of my pair programming partners could tell you, I have the annoying habit of saying “Stop thinking” during refactoring. I’ve always known this isn’t exactly what I meant, because I can’t mean it literally, but I’ve never had a better explanation of what I meant until now. So, apologies y’all, here’s what I wished I had said.

One of the challenges of refactoring is succession–how to slice the work of a refactoring into safe steps and how to order those steps. The two factors complicating succession in refactoring are efficiency and uncertainty. Working in safe steps it’s imperative to take those steps as quickly as possible to achieve overall efficiency. At the same time, refactorings are frequently uncertain–”I think I can move this field over there, but I’m not sure”–and going down a dead-end at high speed is not actually efficient.

Inexperienced responsive designers can get in a state where they try to move quickly on refactorings that are unlikely to work out, get burned, then move slowly and cautiously on refactorings that are sure to pay off. Sometimes they will make real progress, but go try a risky refactoring before reaching a stable-but-incomplete state. Thinking of refactorings as horizontal and vertical is a heuristic for turning this situation around–eliminating risk quickly and exploiting proven opportunities efficiently.

The other day I was in the middle of a big refactoring when I recognized the difference between horizontal and vertical refactorings and realized that the code we were working on would make a good example (good examples are by far the hardest part of explaining design). The code in question selected a subset of menu items for inclusion in a user interface. The original code was ten if statements in a row. Some of the conditions were similar, but none were identical. Our first step was to extract 10 Choice objects, each of which had an isValid method and a widget method.

before:

if (...choice 1 valid...) {
  add($widget1);
}
if (...choice 2 valid...) {
  add($widget2);
}
... 

after:

$choices = array(new Choice1(), new Choice2(), ...);
foreach ($choices as $each)
  if ($each->isValid())
    add($each->widget());

After we had done this, we noticed that the isValid methods had feature envy. Each of them extracted data from an A and a B and used that data to determine whether the choice would be added.

Choice pulls data from A and B

Choice1 isValid() {
  $data1 = $this->a->data1;
  $data2 = $this->a->data2;
  $data3 = $this->a->b->data3;
  $data4 = $this->a->b->data4;
  return ...some expression of data1-4...;
}

We wanted to move the logic to the data.

Choice calls A which calls B

Choice1 isValid() {
  return $this->a->isChoice1Valid();
}
A isChoice1Valid() {
  return ...some expression of data1-2 && $this-b->isChoice1Valid();
}

Succession

Which Choice should we work on first? Should we move logic to A first and then B, or B first and then A? How much do we work on one Choice before moving to the next? What about other refactoring opportunities we see as we go along? These are the kinds of succession questions that make refactoring an art.

Since we only suspected that it would be possible to move the isValid methods to A, it didn’t matter much which Choice we started with. The first question to answer was, “Can we move logic to A?” We picked Choice. The refactoring worked, so we had code that looked like:

Choice calls A which gets data from B

A isChoice1Valid() {
  $data3 = $this->b->data3;
  $data4 = $this->b->data4;
  return ...some expression of data1-4...;
}

Again we had a succession decision. Do we move part of the logic along to B or do we go on to the next Choice? I pushed for a change of direction, to go on to the next Choice. I had a couple of reasons:

  • The code was already clearly cleaner and I wanted to realize that value if possible by refactoring all of the Choices.
  • One of the other Choices might still be a problem, and the further we went with our current line of refactoring, the more time we would waste if we hit a dead end and had to backtrack.

The first refactoring (move a method to A) is a vertical refactoring. I think of it as moving a method or field up or down the call stack, hence the “vertical” tag. The phase of refactoring where we repeat our success with a bunch of siblings is horizontal, by contrast, because there is no clear ordering between, in our case, the different Choices.

Because we knew that moving the method into A could work, while we were refactoring the other Choices we paid attention to optimization. We tried to come up with creative ways to accomplish the same refactoring safely, but with fewer steps by composing various smaller refactorings in different ways. By putting our heads down and getting through the other nine Choices, we got them done quickly and validated that none of them contained hidden complexities that would invalidate our plan.

Doing the same thing ten times in a row is boring. Half way through my partner started getting good ideas about how to move some of the functionality to B. That’s when I told him to stop thinking. I don’t actually want him to stop thinking, I just wanted him to stay focused on what we were doing. There’s no sense pounding a piton in half way then stopping because you see where you want to pound the next one in.

As it turned out, by the time we were done moving logic to A, we were tired enough that resting was our most productive activity. However, we had code in a consistent state (all the implementations of isValid simply delegated to A) and we knew exactly what we wanted to do next.

Conclusion

Not all refactorings require horizontal phases. If you have one big ugly method, you create a Method Object for it, and break the method into tidy shiny pieces, you may be working vertically the whole time. However, when you have multiple callers to refactor or multiple implementors to refactor, it’s time to begin paying attention to going back and forth between vertical and horizontal, keeping the two separate, and staying aware of how deep to push the vertical refactorings.

Keeping an index card next to my computer helps me stay focused. When I see the opportunity for a vertical refactoring in the midst of a horizontal phase (or vice versa) I jot the idea down on the card and get back to what I was doing. This allows me to efficiently finish one job before moving onto the next, while at the same time not losing any good ideas. At its best, this process feels like meditation, where you stay aware of your breath and don’t get caught in the spiral of your own thoughts.

7 Comments

Joshua KerievskySeptember 19th, 2011 at 7:32 pm

Very nice explanation Kent. I like the idea of bringing dimensions into the language of refactoring. It fits well with other terms we’ve been using, like parallel change and narrowed change.

AlessioSeptember 20th, 2011 at 10:07 am

Basically each refactoring must be atomic. “Thinking” during a refactoring is like starting a new thread. If you want to “think” during refactoring do so in the “dispatch thread”!

J. B. RainsbergerSeptember 20th, 2011 at 12:24 pm

Whatever you’re working on, when you want to focus, have a card and pen handy. When something pops into your head that you don’t absolutely have to do right freaking now, write it down in 5 words or less, then get back to what you were doing. It works wonders.

What makes this work? If your tasks take an hour or less — or perhaps even 30 minutes or less — then nothing will sit very long. Almost anything that comes to your mind can wait 30 minutes. You’ll get to it. In the meantime, you got /this/ thing /done/, and that matters.

Ablerto FunariSeptember 20th, 2011 at 5:34 pm

I like to carry a pencil specificatorially to commence field rework. I can do this in any transport mode without disturbancing to the equilibrium. After the completion, it is easy to apply the outcomes to the code body and result with the new structures.

Tim LesherSeptember 21st, 2011 at 4:09 am

“Keeping an index card next to my computer helps me stay focused. ”

Out of all the techniques in “TDD by Example”, this one has proven the most useful to me, and it’s not even about TDD.

It helps me keep my brain at a single level of abstraction–anything higher or lower goes on the list and out of my short-term memory.

Derek WilliamsSeptember 26th, 2011 at 6:00 pm

Thanks for coining a natural pair of terms to describe the way we (usually subconsciously) think when refactoring.

And PHP?! OK, I’ll admit, with a little surgery and lots of care, PHP can be OK. Besides, it’s usually the design, not the language, that truly matters.

ShmooSeptember 30th, 2011 at 10:34 am

Hi, Kent,

Here’s another experience, called, “Fluff-driven development (FDD).”

Prepare yourself by buying some fluff from your local hardware store; it’s usually sold by the plateful. Have your plate of fluff beside your laptop, and have in mind an application you want to write.

The steps of FDD are simple:

1. Take a tiny piece of fluff from the plate and put it on your head, holding your head quite still to ensure that the fluff does not fall off your hair.
2. Write a line of code.
3. Say, “I am the Fluff Lord, within the Dominion Of The Fluffists.”
4. Repeat.

That’s it.

Seriously, that’s all there is to it.

If at any time you even THINK about writing a line of code before putting fluff on your head, then you’ve to delete all your code, shake all the fluff from your head onto the plate and start all over again.

You must be strict in this point! If you think of writing code without your fluff routine in place, then you will only write bad code and it will be filled with bugs. Obviously.

I have noticed that highly experience programmers have problems with FDD; many of them say FDD is a waste of time. Novice programmers, on the other hand – especially those who have paid for FDD courses – are eager to do FDD and then sing its praises to anyone who will listen.

Those experienced programmers are wrong; they must be re-educated in the way of the fluff.

I have been practicing FDD for 12 years now; sometimes in the office, I have so much fluff on my head that my boss thinks I’m a hay-stack, only made of fluff. A sort-of fluff-stack, if you will.

But one thing is beyond doubt: the code I write, when I’m in this fluff-zone, is the most high-quality code that anyone has ever seen.

And I write so fast!

I know what you’re thinking: surely, the time taken to put that fluff on your head is time taken from designing code according to established principles; and surely, putting fluff on your head has nothing to do with writing high-quality code. But you would be wrong to think this. Very wrong. Scarily wrong. Fluff has EVERYTHING to do with quality.

And speed!

When you become proficient at FDD, you will find that putting fluff on your head actually makes you code faster. In ways I can’t quite explain, and in ways you won’t understand until you’re telling someone how putting fluff on your head makes you write higher quality code faster, putting fluff on your head makes you write higher quality code faster.

But I’m not married to it. There has been plenty of high-quality code written throughput the years without a single ball of fluff ever being removed from smooth, white porcelain. I’m simply not disciplined enough to write high-quality code without fluff; call it a flaw, but that’s the way it is and I know my limits. And if you know a better or faster way to write high-quality code without putting fluff on your head, then show me! I’ll glady use your process if it’s better.

As I say, I’m not religious about fluff, it’s just the best process I – and many great programmers that I respect – know.

Please try the FDD steps above. In loads of katas. For the next 12 months.

Yours, sincerely,

Shmoo

PS Fluff also helps you write code when you’re tired.

PPS If you’re not using FDD, then you’re living in the stone age. I’m sorry, but it’s true.