Build Fast and Fix Fast


I’m a fan of nightly builds with automated smoke tests, run overnight. In the morning when everyone returns to work, anyone who’s broken the build fixes it. In most cases, the developers see what they did and they fix it. The agile folks take this even further and say to build the system whenever you’ve made a change and verify the changes immediately with your pre-written tests. That works too, and you have to have set up the system initially to be successful with being able to build small pieces that hook into the rest of the product, once the product is substantial.

I recently met some other folks who take 1-2 weeks between builds, and then it takes them 1-2 weeks to settle the software down between builds. This quickly builds up technical debt, and the debt can be so overwhelming that it takes more than half the developers’ time to fix the debt.

Here’s why. When developers have a chance to see why they made mistakes when their reasoning is fresh in their minds (building whenever you make a change, or at the longest, once a day), they have close-to-immediate feedback about their work. The longer the feedback takes (1-4 weeks is a very long time), the developers have forgotten why they made those decisions initially. Fixing problems in large systems becomes more of trial and error rather than systematic fixing, because the developers have lost the context of what they were thinking.

This is another case of multi-tasking (Hal just wrote about this, Esther Derby wrote about multitasking as have I in previous posts. My next SD article is about multi-tasking.)

Software development (or software testing, or project management, or any other knowledge creation work) is context dependent. If you change your context often enough, you can’t remember what you were doing or why. Then you make mistakes.

For builds, make the time between builds as long as you think you can go and still throw away all the work you did for the previous build. So the faster you want the project to go, the more frequently you need to build. If you have a lot of time, or don’t care as much about predictable schedules, you can take a few weeks between builds. (When was the last time that happened??)

If you’re in a tightly scheduled project, build at least once a day. That way if everything you did the day before was garbage, you can throw it out and only lose one day. Building twice a day is even better. Building every time you make any change at all is best. But no matter how you think about it, build fast to fix fast.

Leave a Reply