The Responsive Design Project so far: January ’10

In the Beginning

The second program I ever tried to write was a copy of the Star Trek simulator I had seen at the Lawrence Hall of Science. I had gotten through the BASIC language workbooks in a weekend. I had written a Roman numeral printer with no problems. How hard could Star Trek be?

I can remember my feelings sitting down at the teletype: anticipation, excitement, puzzlement, frustration, despair. I had no idea what to type. I knew the statements of the language, no problem. What I didn’t understand was design. I had no idea what elements there could be above the level of statements or how those elements could be related to produce the behavior I was imagining.

That day I just gave up and walked away. Over the next decade I slowly got a sense of what elements there could be in a program and how they could relate to each other. I acquired an aesthetic sense. I could take pleasure in a good design and feel discomfort in the presence of a poor one. Eventually I could take a problem and design software to address it.

But something was still missing. I couldn’t explain what I did when I designed. Little pieces like refactorings I learned, with a lot of collaboration, to articulate, but I couldn’t escape the nagging sense that design had hidden structure, like the plumbing and wires in the walls, structure that was critical to really understanding the process and product of design.

Revealing that hidden world of design is the goal of the Responsive Design Project. I want to understand and articulate the deep structure of software design. While the project has only just begun, I have some practical results I’d like to take you through briefly in this introductory article. And I’ll share with you the result that most disturbs and excites me:

• Our illusion of control over software design is a dangerous conceit best abandoned.

In the Slightly Later Beginning

In 2005 I was invited to sit on a panel celebrating the 25th anniversary of the publication of Ed Yourdon and Larry Constantine’s Structured Design, the book that introduced the terms coupling and cohesion. I’d had a copy in my library since I’d used it as a college textbook, but, since I had used it as a college textbook, I hadn’t really read it. As I studied it in preparation for the panel, I realized that many of the questions I had about software design had been answered two decades before. This motivated me to make my informal study of design more structured and rigorous, and to communicate the results to a new generation of programmers. This was the official genesis of the Responsive Design Project.

Design is good. Design is central to effective software development. Programmers can add features steadily to well-designed software. Programmers can easily test well-designed software. Well-designed software is easy to tune for better performance. Most of the hard problems in programming turn out to be design problems.

But design has a dark side. While there isn’t a single best design for any system, there are many poor designs for that same system. The more experience a programmer has, the more design ideas he knows. The temptation is to put these design ideas in the system now because you just know you’ll need them eventually. Over-designing early leads to delaying feedback from real usage of the system, makes adding features more complicated, and makes adapting the design more difficult. By the same token, under-designing makes adding features more complicated, increases defects, and makes adapting the design more difficult.

The word “responsive” in the name of the project reflects this need for balance. Some design needs to be done in advance of coding, but over the life of the project most design will be done in response to the changing needs of the system and the changing and growing understanding of the developers.

The project is based on three parallel tracks of inquiry:

• Quantitative. Measure real designs and see how they behave “by the numbers.”
• Introspective. Carefully watch my own practice of design and draw lessons from the patterns I find.
• Pedagogical. Teach software design and draw lessons from the students and their subsequent practice.

The Tour

Here is a quick overview of the lessons I have learned so far. Each topic deserves an essay of its own, or even a book chapter (say, you don’t know anyone who publishes technical books, do you?).

Beneficially Relating Elements. Designs are made up of elements that relate to each other in beneficial ways. Looked at actively, designing is creating and deleting elements, creating and deleting relationships, and increasing the benefit of existing relationships. Elements contains elements and so on down (and up).

Design is Fractal. There is no fundamental difference between implementation, design, and architecture. They are all a matter of beneficially relating elements. Learn to bounce between levels of abstraction. Most design takes place at a single level of abstraction but big advances often come as a result of rearranging the levels.

Safe Steps. Resolve the tension between efficiency and safety in favor of safety, then learn to take small, safe steps fast enough that from the outside it looks like you are flying.

Isolate Changes. Before making a change, isolate the area to be changed from the rest of the system so you can change an entire element at a time. For example, before changing a part of a procedure, extract the area to be changed into its own procedure. Make the change, then inline the changed sub-procedure if appropriate.

Embrace Ambiguity. Half done with a change and not sure what to do next? Stop and deploy (you can do this if you are working in safe steps). Complete the change when you know what to do.

Cultivate Confidence. Master your tools. Your feeling of mastery will improve your cognition.

Cultivate Humility. Try tools or techniques you aren’t comfortable with. Being aware of your limitations will improve your effectiveness.

Exploit Symmetries. Divide similar elements into identical parts and different parts.

Trust Succession. Design simply today, knowing you can add complexity needed tomorrow.

Design Is a Team Sport. The social side of design is as important as the technical side. Learn the skills needed to communicate and understand designs.

Play with Words. Recasting a design using a different metaphor can revolutionize your understanding.

Play with Pictures. Explain a design using hand-drawn pictures. Ask others to do the same. Design benefits from your whole mind: symbolic, verbal, visual, kinesthetic, conscious and unconscious.

Clear Strategies. All design changes use one of four strategies:

• Leap. Make the new design and use it immediately.
• Parallel. Make the new design and run both new and old simultaneously.
• Stepping Stone. Create a platform to bring the desired new design within reach.
• Simplification. Only implement part of the design now and add complexity bit-by-bit later.

Inside or Outside. Change the interface or the implementation but not both at the same time.

Toss It. Checkpoint often. If you get confused, learn what you can and revert. Now. Beware the sunk cost fallacy.

Keep It. If the system is working and creating value, keep it and improve it.

Start over. Sometimes the constraints of the problem plus the constraints of the existing system are just too much to handle at one time. I prefer incremental improvement, but sometimes it’s best to just wipe the slate clean and start over.

Both. Faced with design alternatives without a clear winner, do it every way. Just coding each alternative for an hour is more productive than arguing for days about which is better in theory, and a lot more satisfying.

Suit Design to Needs. A design strategy intended to maximize feedback from rapid changes looks very different from a design strategy intended to increase value by reducing cost. Both are appropriate at different phases of a system’s lifecycle.

Use Tension. There will always be tension between designing better and delivering sooner. Learn to fuel your creativity from this tension.

Conclusion

Many metrics in design are power-law distributed. For example, if you graph a histogram of the cyclomatic complexity of the methods in a large system on a log-log graph you get a straight line. In other words, there are many very simple methods and a few very complicated ones.

This same distribution is found in many measurements from nature—the intensity of avalanches, for example. This leads me to the most exciting and disturbing lesson I have learned about design so far:

• Design is not a rational process.

Much as we like to think we are making design decisions, much of the structure of our systems emerges from the nature of the problem and the nature of the tools we use. Our designs shape us as much as we shape our designs. Insisting that we as designers are “in control” is counter-productive, leaving us wasting energy trying to act against nature. The challenge is to embrace responsibility and at the same time take an appropriate role in designing. Learning to achieve this balance is the next goal of the Responsive Design Project.