What Model Do Your Estimates Follow?

Cone of UncertaintyFor years, we bought the cone of uncertainty for estimation—that is, our estimates were just as likely to be over as under.

Laurent Bossavit, in The Leprechauns of Software Engineering, shows us how that assumption is wrong. (It was an assumption that some people, including me, assumed was real.)

This is a Gaussian (normal) distribution. It's what we expect. But, it's almost never right. As Laurent says,

“Many projects stay in 90% done for a long time.”

What curve do our estimates follow if they don't follow a Gaussian distribution?

Troy Magennis, in “The Economic Impact of Software Development Process Choice – Cycle Time Analysis and Monte Carlo Simulation Results,” suggests we should look at the Power-Law (Weibull) distribution.

What this distribution says with respect to estimation is this: We are good at estimating small things. We get much worse with our estimation quickly, and for the long tail  (larger and larger chunks of work), we are quite bad.

Why? Because creating software is innovation. Building software is about learning. We better our learning as we proceed, assuming we finish features.

We rarely, if ever, do the same thing again. We can't apply precise estimation approaches to something we don't know.

You should read Troy's paper because it's fascinating. It's well-written, so don't worry about not being able to understand it. You will understand it. It's only 10 pages long.

The question is this: What effect does understanding an estimation model have on our estimates?

If we know that the normal distribution is wrong, then we won't apply it. Right, why would you do something you know to be wrong? You would not estimate large chunks and expect to have a +/- 10% estimate. It doesn't make sense to do that.

But what can we do? On the printed paper, what the proceedings will show p. 5061, Troy has a table that is telling. In it, he says that if you have large unique work items or you have large WIP, you will have poor predictability. (He has suggestions for what to do for your process.)

My suggestions for your estimation:

  1. Estimate small chunks of work that a team can complete in a day or so.
  2. Keep WIP low.
  3. Replan as you finish work.
  4. Watch your cycle time.
  5. No multitasking.

What should you do when people ask you for estimates? What kind of requirements do you have? If you have large requirements, follow my advice and use the percentage confidence, as in We Need Planning; Do We Need Estimation? Read the estimation series or get Essays on Estimation.

You can predict a little for estimates. You can better your prediction. And, you may have to predict a large effort. In that case, it helps to know what distribution model might reflect your estimate.

3 Replies to “What Model Do Your Estimates Follow?”

  1. Producing software is *invention*. It’s hard to invent on schedule.

    Plus engineers tend do give you an estimate for the time to “do” something rather than the time it takes to “Design”, “Code”, “test”, “Deploy” something.

    I’ve taken to starting with their “do” something estimate, and then asking a bunch of “what about this step?” questions, then adding buffer. Then tracking where we’re at on a weekly basis, and where we need to eat into the buffer, we do, then we have a non judgmental discussion about how much to adjust future buffers.

    I often give people the analogy that they didn’t drive to work by pointing their car in the right direction, shutting their eyes, and hitting the gas. That wouldn’t work. You have to steer, with your eyes open to get to work.

    Expecting a project to arrive by pointing and closing your eyes is doomed to failure.

    1. HI Pierce, I do like the idea of invention. I still think that innovation or learning covers invention, but maybe your word is better. I’ll have to think about it.

      Your idea of asking “what about this step” is exactly the kind of thing I wrote about in How to Use Inch-Pebbles When You Think You Can’t.

      I really like your analogy about driving to work. I’m going to steal that one!

      Risk management is about planning and replanning (and replanning…). If you’re serious about projects, you replan.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.