Variance: Lessons from Poker 2
One of the great things about styles of software development is that two people can hold absolutely contradictory opinions and they can both be right. Context is critical to what makes sense. (Hence the futility of searching for “best” practices.)
I was thinking about this today when thinking about our style of development for JUnit. I had tweeted that Saff and I were making “aesthetic” improvements. That’s how we work–we will add a few features, then spend a (seemingly) long time “cleaning up”, establishing symmetries, normalizing names, extracting and inlining components. At some point, we see the opportunity to add new features and progress is rapid.
Our style of development on JUnit delivers uneven results when viewed from the outside. We can go months without a new feature. Sometimes when we add features, though, it feels like we’re wearing jet packs.
One of the concepts in poker is variance, the magnitude of the swings suffered simply because of the effects of chance. Different strategies result in different amounts of variance. One strategy might result in small but steady winnings. Another would result in larger winnings but larger swings. The strategy with greater variance would only be appropriate for an emotionally stable person with a large bankroll.
Our JUnit strategy is high variance. High variance is not appropriate for all projects. If you’re making external commitments, you should work in a more predictable way. Building and maintaining trust is more important than making occasional blazing progress.
We have the equivalent of a large bankroll because we make no external commitments beyond promising to continue improving JUnit. We enjoy improving the internals of JUnit as much as we enjoy releasing new features, so we aren’t bothered when there is a long gap between features.
How should you develop? As always, the answer begins with “it depends”. One of the factors it depends on is how much variance you are willing to support.
I disagree that the search for best practices is a futile exercise. It’s only futile if all you’re interested in is the destination, not the journey.
Even in your example, there is a *best* practice when given the particular context.
I agree absolutely, from the other end of the spectrum. My current project is all about reducing variance, because the unpredictable time required to add a feature was not acceptable. So one rewrite later, I’m proud that the developers can state how long a new feature will take to add.
I agree that it’s possible, reasonable, even admirable to search for better practices. I disagree with the use of the word “best” (in spite of having authored a book with “best practice” in the title). I think “best” is most often used as an attempt to avoid responsibility, as in, “You can’t complain because we were using best practices.” I prefer to use a word like “my” instead. I think it more honestly captures what people mean when they say “best practice”.
I disagree that we use “best practice” in developing JUnit. We have a style that meets our needs, but that doesn’t mean there isn’t another style that would be better in every way. We don’t deliberately work sub-optimally, but applying a word like “best” to our style of development is unscientific.
When I woke up this morning I realized that there are many different high- and low-variance development styles. If I get time today I’ll try to catalog the ones I know about.
Kent, one could write a whole book about how important it is to consider context when deciding your development strategy. Perhaps you already have!
Obviously, one of the main contextual factors in software development is whether you are developing it for free or for a commercial enterprise. There are great benefits and great constraints on both sides.
Commercial enterprises often demand a lot of predictability, and the demand for predictability often leads to smart people reasonably choosing a strategy that focuses on short term results. Often, the same practices that lead to good short term results also lead to good long term results, especially when you consider non-engineering benefits of delivering quickly, such as being able to test out bad ideas in the marketplace, or reduce financial debt, or reduce operational debt.
Still, there is inevitably some tension between short term results and long term results, and your example of how you develop with Saff serves as a perfect example. So it’s probably very liberating to work on a free software project, where you can work in a more bursty fashion in terms of features produced, but you can actually work in a more consistent fashion in terms of improving the internals, which probably leads to greater success over the long term.
Another contextual element, of course, is the problem that the software itself intends to solve. The most cliche examples are, of course, the 747 that absolutely cannot crash or the bank software that absolutely cannot fumble a transaction. But setting aside the tolerance for bugs, you also have a lot of variation within more standard business contexts. For example, if software is tied to some kind of operational concern, the volume of your current operations should greatly drive your methodology. If you have low operational needs at the moment, then your biggest debt worry is probably technical. But if you have high operational needs at the moment, you might be building up operational debt even more quickly than technical debt, and that gets really stressful!
An example of variance is Apple’s “Snow Leopard” version of MacOSX, which had few new user-visible features, but had many small changes:
Speed improvements in existing features,
Small tweaks to improve the user-interface here and there,
reduced disk footprint,
and a new threading facility.
The previous OS, “Leopard” had many new features. Snow Leopard was priced lower because of the lack of new user-visible features.
Most commercial software often has a lot of pressure to just keep adding new features every year, which can cause increasing technical debt if there isn’t enough refactoring going on. It sometimes seems that the more financially successful a product is, the more pressure is exerted to use poor development practices to develope succeeding versions.
This isn’t directly on the topic of variance, but it speaks to some of the same themes that have come up in the comments:
http://jacobian.org/writing/the-power-of-no/
In a nutshell Jacob talks about how writing free software (Django is his example) removes the commercial pressure to keep adding new features. To bring it back to the original topic of variance, the Django project also has lots of constraints that other projects do not have. Because it fairly popular, they have to be extremely attentive to backward compatibility issues. Because their community consists of volunteers, they have project management challenges in terms of motivation–not that “motivation” is easy anywhere, even with a paycheck involved. Because they originally developed the project in a fast-paced newspaper environment, in my opinion they have some pretty serious technical debt.
This is a really interesting analogy. Actually, it also helps a lot, as a user of JUnit for a few years now, to understand your flow of things. You should have told me this when I complained that a patch takes years to be accepted!
Another lesson in this to gain from poker is that you can only act on the information you’ve got, and that even doing the right thing won’t always work.
Consider this: imagine you’re playing 5 card draw. Once your hand is built, you’ve got a straight flush. Naturally enough, you bet heavily – going all in – and this is the right thing to do. What you don’t know is that your (equally heavy-betting) opponent has a royal flush.