CD Survey: How often do you run unit tests?

Running functional tests generally takes longer than running unit tests, so as expected, most people run unit tests more frequently than they run functional tests:

How often do you unit test?

I was a bit surprised at how many people are running tests on every change, but that’s good news for JUnit Max, which shaves seconds off the delay waiting for feedback from tests. If those delays happen a hundred times a day, the cumulative effect of a tool like Max can be substantial. I’m not sure what to make of the teams that run unit tests daily or weekly. I’d like to talk to such a team to understand more about their process.

Unit testing wasn’t explicitly addressed by any of the presenters at the Continuous Deployment webinar, I think because unit testing is assumed when you’re deploying on a short cycle. I don’t recall who said it first, but the model I use is that the defects leaving any stage are proportional to the defects entering that stage. No matter how good your functional tests are, defects leaving the programmer’s desk need to be rare if the software is always going to be ready to deploy.

4 Comments

Dave AronsonJuly 6th, 2010 at 1:55 pm

I try to have a few fast top-level functional tests among those I run on every build of a given piece. Frex, say I’m building a module to implement some piece of functionality. On each code change I’ll run the unit tests of what changed… AND the fast tests (vs. slower tests that will be run less often) of anything that depends on them, possibly all the way up to the entire module — maybe even the whole system, if the tests are still fast enough.

Christian HorsdalJuly 7th, 2010 at 12:15 pm

Can you see from the answers whether people run _all_ the unit tests or just some on every code change?

adminJuly 7th, 2010 at 12:19 pm

No. I will publish the raw data when the series is done, though.

DahliaJuly 7th, 2010 at 2:10 pm

I did a similar thing a few months ago.
http://dahliabock.wordpress.com/2010/04/09/your-testing-tendencies-survey-results/

Leave a comment

Your comment