The Debugger as a Design Tool
I use the debugger as a design tool. This comes as a surprise to my programming partners at times, so I thought I would write the technique down. The roots of the effectiveness of the debugger for design go much deeper than I expected. The debugger makes design more efficient by reducing the space of possible changes. Here’s the story:
The Problem
My partner and I were having trouble with responsibility allocation. We had object A pointing to B which pointed to C. We could see that some logic and data in B didn’t belong there, so last week we moved them to C. Turns out this created more problems in the code than it solved. We really needed the responsibility in A:
We had just started to move the responsibility when a pile of tests failed unexpectedly. My partner wanted to immediately examine the code in A, B, and C to see if we could spot the error. I suggested instead that we set a breakpoint and have a look at the stack at the point where the exception was raised. We did, and quickly found and fixed the problem.
Hmmm…
I’ve done this many times in the past, caught errors in a debugger and examined them, but I never understood why it worked so well. Here’s what I think is going on. The basic information provided by a debugger is the current state and the program counter. The program counter is not just the location in the current routine, but also the location in all calling routines:
There is a lot of information implied by what is shown in this picture. Each frame implies history–statements before the program counter have already executed.
If we had a complete trace of every statement already executed, we would be able to partition the statements in the system into those that have already executed and those that have not:
Efficiency
Wherever the test fails, we are going to have to make a change somewhere in the statements that have already executed. This partitioning is the origin of the first level of efficiency of the debugger as a design tool. Rather than look at the entire system, or even some ad hoc subset, the debugger gives you important clues about precisely where to look to make changes. One of the statements in red in the picture will have to change.
This partitioning may account for why fixing tests with unexpected exceptions tends to be easier than fixing tests with failing assertions. By the time an assertion has failed, generally the whole test has executed, leaving a large body of code to examine for the problem. If an exception is thrown part way through the test, the set of already-executed statements will be smaller, sometimes much smaller.
(Perhaps this partitioning also helps explain the value of unit tests in accelerating debugging. If a test executes a tiny portion of the whole system, isolating a fault in that subset is much easier than isolating the same fault in the system as a whole.)
A second level of efficiency of using the debugger as a design tool comes from the observation that much of the time it is one of the routines on the stack that need to be changed. I don’t have data on this besides my own vague observation (“Oh look, we’re changing a routine on the stack again.”) If the probability that a routine needs to be changed really is strongly affected by whether or not it is on the stack, then a reasonable strategy for finding offending code is just clicking up the stack. I generally do this early in debugging, generally much sooner than my programming partners.
Having a small bag of tricks to respond to errors seen in a debugger further improves efficiency. Most problems are fixed by moving logic up or down the stack. So, addressing the design of the system in the debugger benefits from the cross product of reducing the possible places to change, reducing the probable transformations, and the probabilities of these locations and transformations clustering around the stack itself.
Conclusion
The process I’m describing here looks a bit like “just” debugging. It is a design process because the goal is to strengthen the beneficial relationships between elements. When fixes span object boundaries, even though those objects are on the same stack, you are making design decisions. In the relatively rare case where the problem isn’t directly on the stack, but buried in history, the design changes required can be substantial. In particular, if you have an unsatisfied temporal dependency (“Oh, this code should have been called before that…”), you may have to fundamentally restructure the flow of control to include previously-uninvoked objects. While the immediate goal is to remove the defect, the secondary goal is to strengthen the design in the process.
Do this design in a debugger. It provides you with the greatest possible relevant context for your decision. Factor in the unequal distributions of problems (most often found directly on the stack) and solutions (most often involving moving logic up or down the stack) and you have a technique that on average quickly isolates and remediates faults while preserving an overall view of the design of the system.
The next step of this research is to confirm the biases that I hypothesized. If validated, they could provide the basis for even more efficient tools for debugging and improving design at the same time.
Really love this perspective. It reminds of something that David Ungar once said which goes along these lines: “In a dynamically typed language you get a to play with a real system of object which you can really pound on until you get them do the thing that you need”.
Wish I could find the exact quote…
Using the run-time stack to reduce the set of statements you need to consider is essentially a form of dynamic backward slicing — I’m sorry I don’t have time right now to track down up-to-date refs. but this paper discusses that and some related techniques briefly (look to it’s references for more): http://www.citeulike.org/user/creswick/article/1084566 (pdf: http://csce.unl.edu/~ruthruff/papers/jvlc05-faultloc/JVLC05-FaultLoc.pdf )
This is fairly trivial in data-flow languages, since the “stack” is very clear, and it’s also representative of the entire set of “things” that can affect the code at the given location of the program counter. However, in traditional languages it’s often more complex: obviously the current stack is not sufficient if there are mulitple threads (well, you have multiple stacks, and likely shared memory) and there are also constructions that could be passed in to the current stack that were non-trivially created by other (now terminated) method invocations.
I do think it’s about time for a new paradigm of debugging. Test selection, generation, instrumentation and continuous integration systems could come together to provide a very powerful interface for navigating historical program traces and comparing stack frames across versions of a program to determine the source of errors.
Couldn’t agree more. It’s always saddened me that debugging and testing are often presented as enemies that are mutually exclusive. A good developer should use them both, a point I made a while ago in several forms:
http://beust.com/weblog/archives/000055.html
http://beust.com/weblog/archives/000202.html
[...] Three Rivers Institute » Blog Archive » The Debugger as a Design Tool http://www.threeriversinstitute.org/blog/?p=348 – view page – cached #Three Rivers Institute RSS Feed Three Rivers Institute » The Debugger as a Design Tool Comments Feed Three Rivers Institute Welcome to JUnit Max To Design or Not To Design? A Third Good Question — From the page [...]
I can’t quite agree with the title. What you describe is how you use the debugger to locate a bug. Ahem, that’s what debuggers are for.
A design tool, in my book, is something that helps you design and write code before you decide to try it out.
In my experience, most design happens after you write the code and try it out. That’s when the debugger becomes a valuable design tool.
I do not have a lot of experience in development yet, but the one of the “issue” I observed amongst my fellow junior developers is the fact that most of them doesn’t really use the debugger that often and, as a consequence, are not skilled at using them (therefore creating some kind of “evil loop”; they use the debugger less and less and they are less and less efficient with it).
It has become for me a way to evalute the skills of a developer; how good is he at debugging? how fast is he at finding a bug using a debugger? Usually, people with good debugging skills are also quite efficient at writing “good” code (but maybe it is only because they’re more dedicated to development).
Otherwise, it seems true for me that the use of a debugger can help in your design by giving you another point of view, rather than just looking at plain code; having an idea of what’s in the context of your application at some point can help you to easily check that the code is behaving the way you want it.
Brilliant piece of text. I would like to think of it as follows.
In the automotive industry a lot of effort is put in making cars safe and even safer. But crash tests are the proof of the behaviour of the designed safety precautions built into a car. The results of the crash test show how the components behave. Video caps and sensor data are used to analyse the movement of parts in the car.
Walking through the vid caps and sensor data will give engineers information on how to redesign or recreate parts.
This is exactly what debugging is about in software. Replaying the scenario over and over until you ‘see’ what the problem is. You then decide to redesign or refactor or just fix the bug.
Now, isn’t that beautiful?
Also, I think the debugger is valuable in discovering the design of exiting/legacy code. The source code does not give you the full picture. With a debugger you can see the actual (dynamic) wiring of objects, when this wiring takes place, etc.
A few years ago I worked on extending Sun’s Javac compiler. As I was working I noticed that I am using the debugger to understand the design much more often than the raw source code.
You don’t want a debugger, you want a time machine:
http://weblogs.mozillazine.org/roc/archives/2006/12/introducing_amb.html
A time machine would be great, especially integrated with the IDE. I’d like everything “not yet called” to be filtered out of searches. Then be able to say things like “Stop the execution when this variable gets set to this value.”
Actually, I did some programming on a Nintendo development box, way, way back. It was a lousy system, basically assembler-only. But the good thing about it was it kept the last 64K instructions executed, including effect on register or memory.
Nowadays 64K instructions is nothing, but at the time it was like a time-machine. From the point your program crashed (or where you put your break-point) you could go way back to see how it actually got there.
I wonder if the Java VM could be tricked into doing something like that.
Throughout my career I’ve used the debugger as a tool for building code. I come up with a basic concept of what I’m going to do and I code it out taking sometimes 3 or 4 days to get something that compiles. By that time I’m pretty proud of my infrastructure and what I’ve built but then the real work begins. I have to take my lump of clay and turn it into working code. So where do I begin? I set a break point in the main and or every button handler or other event handler in the app and I start stepping. I started this practice in 1993 simply because it seemed natural to me. What I found was that my colleagues thought I was crazy and that I was unnecessarily wasting lots of time in the debugger. After my 3 days of getting something to compile I might then spend 4, 5, 6 hours stepping through the code until my lump of clay had become my “working” masterpiece. Colleagues would point to the 6 hours of time as wasted time and I would point out that I actually only spent about a half hour, maybe less, stepping through the entirety of the source code. So what was I doing during the other 5 hours and 30 minutes? Finding and fixing bugs! Finding and fixing flaws in the DESIGN that only became apparent to me once I began stepping! And the debugger took me directly to these flaws! I didn’t have to waste hours and hours tracing numerous issues upstream! I didn’t have to find the issues after release into production and not only cause myself many many hours of grief but also the end users! How eloquent, simple, and fast!
Later on when Kent and others began writing books on XP, I was introduced to the concept of unit testing. What I began to realize at that time was that using the debugger was great on the first pass, but when you had to deliver lots and lots of versions it became very inefficient to have to re-visit the debugger. I then came across numerous colleagues who would tell me that I was wasting my time in the debugger that I should be writing unit tests and those unit tests would solve all of my problems. I began writing unit tests fairly heavily on larger projects but something wasn’t right if I didn’t spend time in the debugger too. I knew that it was necessary but was just too busy to worry about why. I think the answer is that I was designing on that first pass. I design in the debugger minimally on a first pass and often times even further along than that. The unit tests were necessary to reduce the amount of time I would have to spend in the debugger on successive passes and as the code grew this became more important. I might step through the debugger only in new pieces of code or rarely in older pieces. However, the unit tests would fail and catch huge errors caused by minor changes without me having to retrace through every line of code I had ever written on the given project. I find that use of the debugger while coding, as opposed to when debugging has saved me more time than any other tool in my skill set. Unit testing is second. I need all that extra time… to spend with my kids.
If there’s anyone else out there whose experienced this I’d love to know because I usually feel pretty alone.
Kent wrote:
“Most problems are fixed by moving logic up or down the stack.
…
solutions (most often involving moving logic up or down the stack)…”
Wait, what?
Do you mean fixing bugs involves this, or that fixing the design does? I can kind of see that moving responsibilities between two classes that call (methods on) each other corresponds to “moving logic up or down the stack”.
A concrete example would help me understand this point.
George,
I’m afraid I don’t have an example easily to hand at the moment. Sometimes fixing a bug requires just making a change to code: “<” becomes “<=” and the test passes. Sometimes, though, fixing a bug requires a change to the design first: I’m performing this conditional in B, but I need to perform it in A before B ever gets called. In this case I divide the fix into two phases, changing the design and then fixing the bug. The debugger and in particular the stack helps me in the first phase.
I’ll try to catch myself the next time I do this and write it up in detail.