There’s a fascinating email thread started by David Anderson about What would Agile Auditing Look Like?. Part of the discussion stems from what the definition of an audit is. Audits are about compliance to a defined process. Do we need audits? Sure, for some projects. I would very much like to know that any project in a regulated industry has a defined process and that the project stuck to that process, unless they could show why (with data) a change improved the process. And, if an organization is going to claim a particular CMMI level, or some other “official” process adherence, the only way to really know that is with an audit. But most projects don’t need audits; they need assessments.
Assessments (at least, when I do them) are about looking at what’s working–and what’s not working. If part of the (either defined or implicit) process isn’t working, it’s my job as an assessor to learn about the root cause of the problem, and to suggest alternatives for the people in the organization to consider. When I do assessments where an organization has a defined process, I’ll use that process to see if the projects perform any work like the process. But it’s my job to understand the system of development, not look for compliance to the process. I’ve discovered the causes of problems in projects that an audit would have missed. For example, the variety of architectural pictures of the same system for one multi-site organization was the cause of much of the disagreement and much of the confusion for one large project. Their process didn’t say to have an architectural picture; the developers and architects thought they could use one. One (and only one) would have been great 🙂 And the reason they didn’t have only one picture went back to the management team, not the project team.
An audit may have discovered this problem. But audits are about compliance, which means they tend to blame people. (The best auditors do not blame their project teams.) So when I do assessments, I ask questions such as “What situation or assumptions would have made this a reasonable reaction/decision/whatever on the part of this person?” That helps me see the system of the project, not look for compliance.