The Case for and Against Estimates, Part 1

After the article I referenced in Moving to Agile Contracts was published, there was a little kerfuffle on Twitter. Some people realized I was talking about the value of estimates and #noestimates. Some folks thought I was advocating never estimating anything.

Let me clarify my position.

I like order-of-magnitude estimates. I don’t hire people without either a not-to-exceed or an order-of-magnitude estimate. I explained how to do that in Predicting the Unpredictable: Pragmatic Approaches to Estimating Project Cost or Schedule.  That’s because I have to manage the risk of the money and the risk I won’t get what I want.

Notice there are two risks: money and value. When I need to manage both risks, I ask for order-of-magnitude estimation and frequent demos. When we did the remodel for the house we are living in now—almost a rebuild—we had an estimate from our builder. Our builder was great. He encouraged us to see the house every day to see their progress. The builder was transparent with problems. Was he truly agile? In order for him to create an estimate, we iterated on the designs for each room before he broke ground.

Construction, hardware development, mechanical products—all those “hard” products require iteration on the design before implementation. That’s because the cost of change is so high when you move to physical form.  In my experience, the cost of not iterating before you go to physical form is prohibitive.

So, what is the value of estimation for software? I have said (In Predicting) that software is learning, innovation. We learn from every software project. That makes estimation tricky, if not impossible. Can you estimate? Of course. The problem I see is in the value of the estimate. That value changes for the domain and customer.

If you have a reluctant-to-agile customer, you might want to do more estimation as you work through the project. That was the point of the Moving to Agile Contracts article. You might not even convince a customer that agile is good for them. If you get the value out of working in an agile way, great. You still get the value, even if the customer doesn’t.

Example.AgileRoadmapOneQuarterIf you have a regulated domain or a complex project that you might want to cancel, you might need more estimation as you proceed. I still like using my deliverable-based roadmaps and a not-to-exceed project cost. I would ask, “How much change do we expect?” If the deliverables are going to change every day or week, I don’t see how you can estimate and believe it. You can do a not-to-exceed for a date or cost.

In software, most of the cost is in the run rate for the project.

The image here is an example one-quarter roadmap from Agile and Lean Program Management. In a program, people often need to see further into the future than a backlog or two. I often see organizations requiring six-quarter roadmaps. That’s fine. The roadmap is a wish list. Why? Because it’s not possible to provide a reasonable estimate of that much work that far out without doing some work.

Here’s the tricky part: how much work do you need to do for estimation? I don’t know.

StagedDeliveryIn the Twitter conversation, Glen Alleman mentioned that Raytheon is doing a project using agile. I am pretty sure the agile Raytheon guy I know (socially) is on that project. Yes, they do 4-week iterations. They work feature-by-feature. I believe, although I am not positive, they started that project with a significant investigation period. To me, that project looks a lot more like staged delivery. On the other hand, does it matter??

Does that mean it’s not agile? Well, staged delivery does not require the same transparency and culture change that agile does. On the other hand, does it matter? Remember, I am not religious about what you do. I want your project to succeed.

So what about #noestimates? How does that figure into this conversation?

Here are times when you might not want to bother estimating:

  • You have a fixed target. Management said, “We need this project done by that date.” In that case, get a ranked backlog and get to work. Why fight with people or waste time estimating when you have no idea what you can do? In Predicting, I say something like this, “Get a ranked backlog. Get to work. Get some data. Show progress. If you can’t deliver what they want when they want it, you now have data for a discussion.”
  • You think things will change every day or every week. Management/your sponsor says, “Here’s the ranked backlog. We want to see what you can do so we know what we want to change.” Inviting change is why we use agile. Otherwise, we could used staged-delivery. Why estimate? I would use a not-to-exceed date or cost.
  • You are on the project to save the company. Get a ranked backlog and get to work. Determine how often you can release to get revenue.

I have been on all these kinds of projects. I have gotten a ranked backlog and gotten to work. I have succeeded. Oh, in one case, the company management started the project to save the company too late to make a difference. I didn’t succeed then. We needed four weeks to make a difference and had two.

I like delivering small chunks often. Yes, I use deliverables in my work that are small, often an hour or less.  I can stop when I get to the end of them and not worry about the next chunk. I am sure I do different work than you do.

That is why, as Glen says, the domain is critical. I think it’s also the customer. Maybe there are more things to consider.

In my next post, I will discuss when estimates are harmful.

Tags: , , , ,
Previous/Next Posts
« »

8 Comments

  1. Glen ALLEMAN

    Just checking. Order of Magnitude Estimate in our domain is a 1, 10, 100 estimate.
    These might be useful to test the feasibility of a product or project.
    “Can we have humans fly to Mars using the current technology as proposed by Elon Musk?” The answer by the way is NO.

    Not sure I’d be hiring anyone who can only produce an Order of Magnitude estimate

    Reply
    • johanna

      Huh. Well, that’s why the domain matters.

      I wouldn’t use an agile approach to do space exploration, not at first. I would do a not-to-exceed exploration/discovery project. Maybe several of them.

      I have never worked in an arena where money was no object. I will be talking about how to do more specific estimates in Part 3 (not yet published as of this time).

      Reply
      • Glen ALLEMAN

        Agile (Scrum with SAFe) is used in space exploration. Money is fixed, requirements emerge. These are “engineered to order” capabilities for performing the mission.

        Order of Magnitude = 10X indpeendent of domain. It’s the value in the exponent of 10^X where the X is the order
        10^1 = 10
        10^2 = 100
        10^3 = 1000

        OOM estimates are useful to anyone with that range of outcomes

        Reply
        • johanna

          Glen, if money is fixed, that’s working to a target. Makes total sense to me, and what I tried to say in this post. Maybe I didn’t explain it very well.

          Reply
          • Glen ALLEMAN

            That “target” as you say is developed through a “basis of estimate” process. It is not set in an arbitrary manner, but build through an estimating process. The BOE is used to establish the contract budget base for the contract to procure the needed capabilities for the mission success.
            The notion of setting a Target in the absence of a Basis of Estimate for that target would not provide the needed credibility to accomplish the mission outcomes. And at the same time, the development processes are “emergent” from the needed Capabilities in the form of SAFe’s portfolio management process.
            The notion of a fixed date – a launch date, for the spacecraft or a product launch and a budget for that launch – can only be credible of those numbers have a credible Basis of Estimate, built up using an estimating process.
            Then those targets represent a reasonable “estimate” that has a reasonable chance of being met.
            Management saying “We need this project done by that date,” can only be credible if it has a credible chance of being the right numbers (cost and schedule) for the needed capabilities. When management makes proclamations or dates or costs without the underlying “estimates” there is little hope of success.

          • johanna

            Glen, you have nailed the proverbial nail. In your last para, you say:

            When management makes proclamations or dates or costs without the underlying “estimates” there is little hope of success.

            I have worked for many more of these managers than I care to remember. Either managers estimated on behalf of the teams—which meant the managers did not understand the risks—or managers decreed the date.

            It sounds as if you have worked for managers who were interested in understanding the risks. That’s great.

          • Glen B alleman

            I our domain – software intensive system of system – in space and defense, there are three reasons, programs go over budget and get behind schedule, according to my collegaue who was the Cost Direct at NASA under Michael Griffin.

            1. We could not have known – It’s a science project
            2. We didn’t know because we didnt do our homework

            that accounts for around 50% of the root cause

            3. We don’t want to know

          • johanna

            Ah, I have worked for many managers who wanted to remain ignorant of anything. I coached a VP once who automatically cut his people’s estimates by 20%. He thought he was “helping” by creating a sense of urgency.

            When I did the assessment, the people told me he only had to do that once. They now inflated their estimates by 50% or more. That way, they could have reasonable projects.

            Estimation is tricky. It’s hard to do well. I have spent a fair amount of time on what you might call science projects. We didn’t know if we could do them or not, given the power of the machinery at the time.

            I started to use inch-pebbles because it helped me do my homework.

Trackbacks/Pingbacks

  1. The Case For and Against Estimates: Part 5 – Technology Up2date - […] you’ve been following the conversation, I discussed in Part 1 how I like agile roadmaps and gross estimation and/or…
  2. The Case For and Against Estimates: Part 5 - - […] you’ve been following the conversation, I discussed in Part 1 how I like agile roadmaps and gross estimation and/or…
  3. What We’re Reading This Week: July 11th | Kanoah - […] The Case for and Against Estimates, Part 1: Johanna Rothman (@johannarothman), head of Rothman Consulting Group Inc, reveals her…

Submit a Comment

Your email address will not be published. Required fields are marked *