CD Survey: What practices do developers use?

The survey I’ve been writing about (raw results here) was intended to give us speakers at the continuous deployment webinar (Timothy Fitz, Jez Humble, and myself) some background on the attendees. I’ve saved the best (most informative) question for last: what practices do attendees use in software development. Here is the data:

What practices do people use?

Some thoughts:

  • Business-based operations metrics. One of the key insights of continuous deployment is using business-oriented metrics to monitor operations instead of the more natural (for programmers, anyway) technical metrics. If you expect 50 sign-ups per minute and the rate suddenly dips to 20/minute after a deployment, it’s time to roll back. The practice is not in common use.
  • Kanban versus iterations. Iterations still dominate, even though the additional flexibility of kanban is a better match for continuous deployment.
  • Pair programming. For all the complaints I hear about pair programming, I would have expected this number to be lower than 25%.
  • Test-driven development. 50% is higher than I would have expected. Adoption of TDD is excellent preparation for teams wishing to deploy more frequently (see my commercial screencasts for more details).
  • Continuous integration. I expected this number to be higher. CI was the first practice from Extreme Programming to spread widely, but, at least among this audience, it is not pervasive.
  • More than 75% of teams test manually before deployment. This is a sensible practice until the defect rate is brought down and the operations infrastructure made robust in the face of errors, but I expect the number to drop as teams mature in their application of continuous deployment.

Change generally happens on a time scale of decades. Mass production and then lean production each took upwards of fifty years to become widespread. I don’t mean to be overconfident, but the picture above (skewed as it is by selection bias) paints a picture of software development that is substantially different than common practice twenty or even ten years ago. There’s still a long way to go until software development pours out the stream of value it is capable of, but we’re making progress.

Commercial plugs: Check out my series of screencasts on intermediate-level test-driven development, $25 for four episodes. If you run unit tests for Java in Eclipse, check out JUnit Max, the continuous testing plugin, $50/year until August 1, 2010.


Ryan CromwellJuly 8th, 2010 at 8:23 pm

“Continuous integration. I expected this number to be higher…”

I am impressed by this number, but as you state this is not an open, random sampling of the industry. While I’m confounded at each client I see not eliminating the wasteful repetition of manual build, test and deploy I am continually confronted with greater than 50% (possibly closer to 25%) of clients without a single automated build. Automated! Not triggered by an otherwise natural development activity, but simply automating a build, test, and packaging of their product is seen as too much effort for too little gain.

Udo WalkerJuly 8th, 2010 at 11:28 pm

“Continuous integration. I expected this number to be higher…”

I suppose this could be easily higher if most of the open source projects would use it to show that it is useful. But even most of the well known open source project hosters do not have the infrastructure for continuous testing of their hostet open source project. Just to mention the lack of this infrastructure on Goolge code, sourceforge,

Michael MahlbergJuly 9th, 2010 at 2:15 pm

“Continuous integration. I expected this number to be higher…”
I just don’t believe it – not that it’s not higher, but that it is so high.
What happened to the original idea as stated in the works from the end of the last century? Since you (Kent Beck) are one of the original proponents you might know better, but I perceived the “integration” part of CI as almost ACID compliant – and at the end end of one such integration the mainline had to be ‘clean and green’ or the integration had to be “rolled back”.
From this perspective every monitoring tool to display the state of the Build Server (cool as the may be) is an indicator of non-CI to me. (Just to be clear: Monitoring the state of the deployed system for all “stages” on the other hand is a good – but under employed – practice in my book

Nuno LopesJuly 28th, 2010 at 12:10 am

We use Automated Builds + Source Control, Manual User Testing, Iterations, Production monitoring assisted by developers.

Continuous integration is a global IT effort if not enterprise wide.