Comments on: When is Continuous Integration Not?

By: JÃ¼rgen Ahting

JÃ¼rgen Ahting — Tue, 11 Apr 2006 14:41:59 +0000

CI does not only expose problems in the code but also in the design and the process.
A few weeks ago I went to a presentation where developers from a company shared their experience with CI. For them there was no question of going on with it, but it was not clear cut that they really saved time in the end after taking into account all their efforts and troubles.
It became pretty obvious that almost all their negative experiences came down to two root causes: bad design and breakdown in developer discipline. And they almost never attacked the root causes but worked in the symptoms only – sometimes making matters worse. Examples: Builds took too long, because modules had too many dependencies and unit tests where often really acceptance tests; Builds broke too often, because developers didn’t run a local build before check-in – they just got accustomed to let the automated CI find any problems – and because of occasional race conditions (they chose to disregard the module dependencies to speed up the build).
CI applied to a system not designed for CI like unit tests added to a system not designed for testing, may result in more effort than gain – but that is not CI’s fault.

By: Johanna Rothman

Johanna Rothman — Wed, 05 Apr 2006 22:27:33 +0000

Lasse, ooooh. The part that gave you chills–that interpretation never occurred to me. Duh.

By: Lasse Koskela

Lasse Koskela — Wed, 05 Apr 2006 18:13:45 +0000

Johanna, something about “the PM seeing every day if the build is broken” gave me the chills. The way I think of CI, it’s for the developers, not management. It’s the developers who see all the time (not just every day but literally all the time) that the build is “green” and are naturally driven towards bringing it back to green when it breaks.
In other words, I’d rather talk about the *team* being all the time aware of the health of their codebase.

By: Simon Baker

Simon Baker — Wed, 05 Apr 2006 14:51:23 +0000

As Martin Fowler says, CI makes the daily developer routine take a fundamental shift to the whole development cycle. A ten-minute build automated with continuous integration helps developers establish a rhythm as they develop software. This rhythm reminds developers to integrate regularly. More about this in my blog post, Ten-minute build, continuous integration and developer rhythm. I coach developers to integrate as often as possible during the day, ideally every couple of hours. To achieve this, I coach them to slice a user story into a collection of vertical slices of customer-visible functionality. When a slice is done, it can be checked-in and integrated, and if successful, CI deploys it to an environment. This allows the customer to see and play with emerging functionality and promotes early and often feedback. The longer a developer refrains from checking code into the version control repository, the more his local workspace diverges from the code trunk. This has inherent risk and is likely to result in a painful merge. Integrating often significantly reduces this problem. Also integrating more frequently reduces the amount of change each time going into the version control repository. If there are bugs they should be easier to find because thereÂs only a small amount of new or changed code to check. Running all the units tests (and acceptance tests, if it can be achieved in a 10-minute timeframe) on each build provides a full regression test cycle every time. Testing doesnÂt necessarily prove that there arenÂt any defects but CI does greatly reduce the defect count. A broken build should not be tolerated, but it does happen especially if developer discipline fails around the definition of ÂdoneÂ. I like to do a local build before checking in. A broken build, however, is a natural outcome for an integration and so the team is responsible for fixing it immediately. There should certainly never be a broken build at the end of the day. In a self-organising agile team, itÂs their tool and their responsibility. If the build is breaking all the time, then thereÂs a problem with the way the developers are working. The ScrumMaster or PM should watch for this. IÂm not sure about using it as a metric, although an operating CI (that is running FIT tests) generates the data to track running tested features, which is what IÂm interested in. CI removes the unpredictability of big-bang integration. By integrating often, integration is broken down into many small integrations that are performed as part of the cycle: test-code-refactor~~integrate. The integration nightmare goes away and the number of integration problems are reduced. This gives developers the courage to move quickly and with higher productivity.

By: Ed Gibbs

Ed Gibbs — Wed, 05 Apr 2006 08:30:15 +0000

One really nice feature is it forces developers not to rely on cryptic build processes. The build has to be scripted out in ant/maven/rake, etc. Before we put in continuous integration we had nasty issues with the build being dependent on a single developer and their magic machine.
A quick win is when developers forget to check in some code, but don’t notice because it still compiles on their machine. It’s usually resolved in minutes after the broken build emails go out.
Our continuous build box also runs Checkstyle and Clover. This gives us regular feedback on how well the code is falling our coding conventions and how the unit test coverage is progressing. These metrics are nice for management and also tend to make more transparent just how clean the code is.

By: david takahashi

david takahashi — Wed, 05 Apr 2006 04:25:30 +0000

If the continuous integration build is automated, there is no reason to not use it.
We would love to have 100% successful builds, but we are happy to have CruiseControl tell us we missed something. It allows us to fail fast!
I got a lot of resistance to the idea of continual integration here: for a long time it was ‘his project!’. Now, people still expect others to do the plumbing, but they sure miss the tool when we turn it off.
Continual Integration should provide the organization with solid metrics. Metrics are one thing that can be in short supply.
Continuous Integration is a pragmatic programming practice: it improves the entire process…
Hope this is helpful…

By: Daniel Read

Daniel Read — Wed, 05 Apr 2006 03:47:09 +0000

Welcome back, Johanna.
Continuous integration can be tricky to implement, not only technically, but logistically. Personally, I would question case-by-case whether continuous integration was *really* of benefit to the project, or whether it’s more about helping management to feel good and have a sense of control. If one were to encounter resistance from the technical team about making this move, one would have to judge for oneself whether the resistance is fear of change or of giving up control, or whether there is a real resistance to the overhead that will come with it.
One area of particular concern is whether the source code control system in use is up to the job of supporting team-based continuous integration, which suggests frequent check-in. For example, does the SCC system support private checkout/checkin areas for developers to save and version their work without having incomplete code integrated into the build? Does the SCC system support concurrent versioning? What I’m getting at is that you want to avoid “not-work” messes like the ones described in this blog post:
http://www.developerdotstar.com/community/node/443
If I were a technical lead on a hypothetical project that was looking to implement continuous integration, I would also want to know whether the original project estimates took into account the overhead of continuous integration, daily builds, more complicated configuration management processes, etc. I would also want to know whether resource allocations for the project allowed for a buildmaster or configuration management specialist, or for the fact that, absent a buildmaster, me or my team members are probably going to be dealing with it.
I don’t mean to be down on continuous integration, but it does not come without overhead, and it’s not a silver bullet. It’s well established that integrating too late and not often enough can be costly, but so can integrating all the time.
Best,
Dan