Program sponsors and the teams themselves want to know where they are in the program. What’s started? What’s not? How much more remains?
It’s tempting to measure “everything.” That creates another problem: You might spend too much time attempting to gather data that doesn’t provide enough information. You might even miss seeing the real progress if you spend too much time attempting to gather surrogate data.
Agile provides us the opportunity to measure what has actually occurred: empirical data. Given the observations, agile program managers don’t need too many measurements. Here are three that will help you see the program’s progress.
Measure What You Want to See
You have any number of possibilities for what you could measure. I have a guideline: Measure what you want to see more of and maybe what you want to see less of.
I want to see more features and see less work in progress, so I measure features and WIP. Because the teams are agile, I expect them to measure their escaped defects, but I might measure those, too—just in case defects are escaping everywhere.
I don’t measure velocity or cycle time because this is a program. Any given team’s velocity or cycle time is unique to that team, so it doesn’t make sense to measure velocity or cycle time at the program level. (I do recommend each team measure its velocity or cycle time to understand how that team’s throughput works for that team.)
But, at the program level, I want to see how many features we’ve completed, where we are with respect to the anticipated roadmaps and how our throughput is. Those are the next three charts.
Measure the Completed Features
Our customers buy or use features, which is why I count features. This chart has three measures: completed features, the features remaining and the total number of features. I count the features; I don’t try to weight the features in any way:
That top green line—the total number of features—continues to rise through most of the program. My experience is that the number of features will change throughout the program, mostly rising. The red line going up is the burnup of features complete. The blue line trending down is the number of features remaining.
Too often, the sponsors or product managers don’t recognize what happens when they ask the product owners for more features: The total features increase, and the features remaining increase. This chart clarifies that and helps them see the effect of their additional feature requests.
Measure the Product Backlog Burnup
Sometimes, you want to see the progress of all the features against each other. The product backlog burnup chart shows that information:
This chart only shows three feature sets. The teams make steady progress on Feature Set 1 all through the program. They are almost done with Feature Set 1 by the fifth date. In contrast, the teams make progress on Feature Set 2, but the number of features increases for the fifth date. The teams make progress, but the teams can’t tell they are making progress because the number of features grew. And, no one has started on Feature Set 3.
Measure the Program’s Cumulative Flow
One of the problems in a program is that work can accumulate, in any number of places. Measure cumulative flow to see that:
In this cumulative flow chart, the product owners remain way ahead of the teams by having many features in the “ready” state. The developers have an uneven throughput (see dev-unit test). The testers are able to keep pace but never quite catch the developers in sys test. The product owners do not accept stories fast enough, which is why the accept area grows.
Once the teams realized what was going on, they were able to manage their WIP, and close the gaps between the states:
By the end of the program, the product owners had caught up with the teams and accepted all the stories.
Consider Other Measurements for Your Program
Your program might need to measure build time to make sure the product builds fast enough. You might need to measure run rate for all the teams to track labor cost. You might want to measure the speed or reliability of various scenarios as the teams develop the product.
Measure the Minimum
Every measurement has a time burden associated with it. The more you measure, the more time you and the teams spend in collecting and reporting the measurements. That means less time creating the product. Instead of measuring “everything,” consider the three or four measures your program needs. Measure and report on those.