CD Survey: Deployment Frequency
In conjunction with the Leaders of Agile webinar on Continuous Deployment I hosted yesterday for SD Times, I ran a poll asking attendees about various of their development activities. I’ll be posting a short piece about each question, the answers we got, and the webinar material that connects with that question.
My co-presenters were Timothy Fitz of IMVU and Jez Humble of ThoughtWorks. Timothy talked about continuous deployment in a large web/client application with ~50 developers while Jez covered CD for IT applications. Both made a nice contrast to my experiences with CD for individual or pair projects, and immediately addressed the “sure, but it won’t scale” objection. My thanks to Timothy, Jez, and Alan Zeichick of SD Times for their work setting up and delivering the webinar. (I’ll post here when a recording is available.)
For the poll, the questions were posted two days before the webinar. We had ~200 responses out of the 1100 registrants and 430 attendees. Beware of extrapolating too far from the data because the poll suffers from both selection bias and an amateur question author (me). However, the responses do provide some idea of how at least a slice of our industry works.
The first question I will cover from the poll is, “How often do you deploy to production?” That’s the fundamental measure of continuous deployment, at least as narrowly viewed (more about that later). Here’s the raw data:
Deployment cycle starts with the orange slice for annual deployment at the bottom right. Going clockwise from there, the three biggest slices (quarterly, monthly, weekly) take up most of the chart. Completing the circle, there are almost as many people deploying on every commit as deploying annually, so continuous deployment already has significant traction.
One of the surprises of the webinar was Jez Humble’s flexibility with regards to how far towards real deployment Continuous Delivery (his phrase and the title of his forthcoming book) needs to go. His strategy is to take the rapid iteration that is made possible in development by Extreme Programming and gradually extend it down the deployment pipeline, even if that stops short of real actual live deployment with real actual live users. My general preference is to work the other way around. I like to get the whole team–business, development, and operations–working with the same rhythm, then gradually increase that rhythm. There’s no right or wrong approach, it’s just interesting to see the contrast.
Next question: how often do you run functional tests?
Commercial plugs: Check out my new series of screencasts on intermediate test-driven development. If you run unit tests for Java in Eclipse, check out JUnit Max, the continuous testing plugin.
Hey Kent
I had a great time on the webinar and learned a lot, so thanks for the opportunity.
I think I may have done myself a disservice in the webinar. It’s true that in the book Dave and I emphasize reducing cycle time over increasing frequency – and as you say, I think both approaches can work. I wish we had said more about the technique of letting increased frequency drive change, because I think it can be very powerful.
But I strongly believe that continuous delivery should extend to release: two of the eight principles we discuss that form the foundation of the book are “If It Hurts, Do It More Frequently, and Bring the Pain Forward” and “Done Means Released”.
In “If It Hurts, Do It More Frequently, and Bring the Pain Forward” we say “If releasing software is painful, aim to release it every time somebody checks in a change that passes all the automated tests. If you can’t release it to real users upon every change, release it to a production-like environment upon every check-in.”
In “Done means released” we say “Really, a feature is only done when it is delivering value to users… For some agile delivery teams, ‘done’ means released into production. This is the ideal situation for a software development project. ” Admittedly we go on to caveat this, but I _do_ think that there are a few cases where continuous release isn’t achievable – although far fewer than is generally believed.
We do say that you can be incremental about _implementing_ a continuous delivery process – but by that we mean that if you have a delivery (build, deploy, test, release) process which relies a lot on manual steps, you can be incremental about automating it. That’s not supposed to imply that you can be incremental about how far towards release you go – in fact, automating the deployment process could well be the first thing you do, if that’s what’s causing you the most pain in terms of delivery.
it is not easy to interpret the data.
Can you change from a pie chart to a bar chart, that is in order of the questions asked?
On every commit it is a metric from another opera. You cann’t put it on a par with the time metrics. Because time between commits is not determinite. On one our project we commit once per week and make release on every comit.
Good point. This survey is just the beginning, it’s obvious now.