A reader asked me these questions about reliable estimation metrics:
- Would you explain leading and lagging indicators?
- Can I use a burndown chart to see accurate progress?
- Is that burndown chart useful if the team has not yet finished grooming the work?
These are great questions because too many of us think the answers are straightforward. However, they are not.
Leading Indicators
A leading indicator is supposed to use current data to predict the future. In software product development, I like these two leading indicators:
- Throughput. (See Flow Metrics and Why They Matter to Teams and Managers for more details.)
- Demos of running, tested features. (To anyone: the team, a product leader, the customer.)
Why these leading indicators? Because consistent throughput means the team is finishing features. And when teams demo, they create shorter feedback loops for themselves and everyone else.
The reason every manager wants agility is because they want to see that stream of finished features. That allows capitalization, etc.
Here's how to think about throughput and demos.
How Throughput Works as a Leading Indicator
Let's imagine the team normally finishes a story every day or so. One week, All Kinds Of Bad Things Happen: a tester breaks his leg and is out while he gets a cast. A developer's small children give her the flu. The power company cuts the power to the corporate building, and everyone has to find a different place to work.
All of These Bad Things mean the team is now on Day 4 of the 2-day story. Their throughput has gone down. They do not yet know their cycle time, but throughput is definitely a leading indicator.
This only works if the team collaborates on most of the work. Why? Because they spend a lot of time waiting for each other when they work alone. (See the series that starts with How to Use Value Stream Maps to Reinforce Agility & Effectiveness, Part 1 (Expert Teams) to see more.)
How Demos Work as a Leading Indicator
Have you ever been on a team where they stopped doing demos? If so, what happened?
Every time I've seen this, the team stopped finishing features. Worse, they stopped checking in with the product leader. When they finally finished something, the customers did not want it.
All because they stopped demoing.
When teams finish a story every day or so, they can demo every day or so. It's easy and fast.
So we can use demos themselves and the frequency of demos as a leading indicator of project health.
Lagging Indicators
Lagging indicators only offer information after some other event. These are examples of lagging indicators:
- Customer satisfaction (because you have to ship something first)
- Revenue, because you need a product first.
All of that means that we need in-process indicators to explain where we are with any reliability.
Velocity is Not Any Indicator, Leading or Lagging
If you use velocity, which I do NOT recommend, you don't see any of this. All you know is that the team is not “making their velocity.” (That's why I recommend that teams Measure Cycle Time, Not Velocity.)
Velocity is a lagging indicator. Worse, you can only see it lagging once the team finishes a story. Unless you use a Value Stream Map and/or many columns on the board, you don't know why the team is “slow.”
As for burndown charts: They are useless. All they do is make people feel bad about what they did not do. I recommend against burndown charts. See Value of Burndown and Burnup Charts for one post.
Why do people use burndown charts? They think velocity should be ever-increasing. But it should not. Read Velocity is Not Acceleration. That post also has the burnup charts I recommend to see more reliable progress. Which brings us to the last question, the one about grooming work.
Grooming Work
When teams use double-loop learning, they often realize they need different stories. They realize they need to “groom” the future work. That's totally fine.
But I suspect the reader's question was more along the lines of this scenario:
- The sprint already started.
- The team has some of the stories they need to do, but not all.
- How does that affect the burndown chart?
Let's challenge all of these ideas because they are the opposite of what Scrum is supposed to be.
If you want to use Scrum, you look at your throughput for the most recent few sprints. How many stories do you finish? Use that number as a guide to predict what you can do for the next sprint. Especially if you right-size the stories.
Relative estimation is useless, as your burndown charts show. But using cycle time to predict? That creates a little more reliability, even if the team works as experts. (Cycle time is excellent once the team cooperates or collaborates.)
Here's my best advice. Ask these questions:
- For the product leader: Assume you can only get two stories from this sprint. Which two do you want the team to finish? What will make the most difference for the customer?
- For the team: How will we work together to finish these two stories?
- For managers: do you care about how the team finishes work or that the team finishes the work?
Too many managers still confuse activity-based measures with outcome-based measures. That misleads everyone about the real progress.
No one can possibly know “everything” the team has to finish until the team finishes something (a running, tested feature), shows it to someone (a demo), and then learns from that.
For More Reliability, Focus on Flow
If you want more leading metrics, use flow as your guide. If you want more reliable estimation, focus on cycle time.
Stop pushing work into a sprint. That forces people to think about work that makes no sense yet. (And might never make sense.) Instead, use “how little” thinking to focus the team on what they need to now.
And, you should read my books:
- Create Your Successful Agile Project: Collaborate, Measure, Estimate, Deliver to see how to make a reasonable agile project for your team.
- Project Lifecycles: How to Reduce Risks, Release Successful Products, and Increase Agility
If you and your managers have not yet read the Scrum Guide, do so. You might also want to read the ProKanban Guide. (Also free.)
Good luck!