We Need Planning; Do We Need Estimation?

As I write the program management book, I am struck by how difficult it is to estimate large chunks of work.

In Predicting the Unpredictable and Manage It!, I recommend several approaches to estimation, each of which include showing that there is no one absolute date for a project or a program.

What can you do? Here are some options:

  1. Plan to replan. Decide how much to invest in the project or program for now. See (as in demo) the project/program progress. Decide how much longer you want to invest in the project or program.
  2. Work to a target date. A target date works best if you work iteratively and incrementally. If you have internal releases often, you can see project/program progress and replan. (If you use a waterfall approach, you are not likely to meet the target with all the features you want and defects you don’t want. If you work iteratively and incrementally, you refine the plan as you approach the target. Notice I said refine the plan, not the estimate.
  3. Provide a 3-point estimate: possible, likely, and worst case. This is PERT estimation.
  4. Provide a percentage confidence with your estimate. You think you can release near a certain date. What is your percentage confidence in that date? This works best with replanning, so you can update your percentage confidence.

Each of these shows your estimation audience you have uncertainty. The larger the project or program, the more you want to show uncertainty.

If you are agile, you may not need to estimate at all. I have managed many projects and programs over the years. No one asked me for a cost or schedule estimate. I received targets. Sometimes, those targets were so optimistic, I had to do a gross estimate to explain why we could not meet that date.

However, I am not convinced anything more than a gross estimate is useful. I am convinced an agile roadmap, building incrementally, seeing progress, and deciding what to do next are good ideas.

Agile Roadmap When you see this roadmap, you can see how we have planned for an internal release each month.

With internal releases, everyone can see the project or program progress.

In addition, we have a quarterly external release. Now, your project or program might not be able to release to your customers every quarter. But, that should be a business decision, not a decision you make because you can’t release. If you are not agile, you might not be able to meet a quarterly release. But, I’m assuming you are agile.

Agile Roadmap, One Quarter at a time In the one-quarter view, you can see the Minimum Viable Products.

You might need to replace MVPs with MIFS, Minimum Indispensable Feature Sets, especially at the beginning.

If you always make stories so small that you can count them, instead of estimate them, you will be close. You won’t spend time estimating instead of developing product, especially at the beginning.

You know the least about the risks and gotchas at the beginning of a project or program. You might not even know much about your MIFS or MVPs. However, if you can release something for customer consumption, you can get the feedback you need.

Feedback is what will tell you:

  • Are these stories too big to count? If so, any estimate you create will be wrong.
  • Are we delivering useful work? If so, the organization will continue to invest.
  • Are we working on the most valuable work, right now? What is valuable will change during the project/program. Sometimes, you realize this feature (set) is less useful than another. Sometimes you realize you’re done.
  • Are we ready to stop? If we met the release criteria early, that’s great. If we are not ready to release, what more do we have to do?

Here’s my experience with estimation. If you provide an estimate, managers won’t believe you. They pressure you to “do more with less,” or some such nonsense. They say things such as, “If we cut out testing, you can go faster, right?” (The answer to that question is, “NO. The less technical debt we have or create, the faster we can go.”)

However, you do need the planning of roadmaps and backlogs. If you don’t have a roadmap that shows people something like what they can expect when, they think you’re faking. You need to replan the roadmap, because what the teams deliver won’t be everything the product owner wanted. That’s okay. Getting feedback about what the teams can do early is great.

There are two questions you want to ask people who ask for estimates:

  1. How much would you like to invest in this project/program before we stop?
  2. How valuable is this project/program to you?

If you work on the most valuable project/program, why are you estimating it? You need to understand how much the organization wants to invest before you stop. If you’re not working on the most valuable project/program, you still want to know how much the organization wants to invest. Or, you need a target date. With a target date, you can release parts iteratively and incrementally until you meet the target.

This is risk management for estimation and replanning. Yes, I am a fan of #noestimates, because the smaller we make the chunks, the easier it is to see what to plan and replan.

We need planning and replanning. I am not convinced we need detailed estimation if we use iterative and incremental approaches.

Tags: , , , , , , , ,
Previous/Next Posts
« »

17 Comments

  1. Jim Grey

    I love all of this and use it as much as I can. I teach three-point estimation to all of my team members, and have done it for 15 years now at various companies, and ask them to do frequent reassessing so I can update expectations — and everywhere I go my management is thrilled with how much visibility I can provide into progress to plan and to problems that might derail things.

    I’ve never worked in a place where I didn’t have to make some sort of projection as to when a large body of work would deliver. But I find that with tools like those you recommend I can give a reasonable estimate, but then update frequently (e.g., every couple days, every week, whatever makes sense) about what we’re experiencing on the ground and what that does to the projected dates. I’ve yet to have an upper management team not just absolutely love that.

    Reply
    • johanna

      Hi Jim. Yes, we almost always have to do some sort of prediction. I have found that if you provide value often in the form of working product, people pay less attention to the prediction and pay more attention to the product.

      I do like asking this, “Would you like us to spend a week providing you a so-called accurate estimate? Or, would you rather we start developing working product and learn from our actions? If we do that for a few weeks, we can provide you a much better estimate/prediction.”

      In agile organizations, that’s a reasonable question. In not-agile orgs, not so much.

      Reply
    • johanna

      Jim, I love that post! Thanks for including the link.

      Reply
  2. Glen Alleman

    Planning is about making decision for future outcomes. “We have a plan for our summer vacation in Italy.”
    Making decision about the future is a probabilistic process.
    Making decisions about probabilistic processes, requires estimating their cost, impact, and potential benefits.
    This is the Microeconomics of decision making and writing software for money.

    Economics of Iterative Software Development is a good start to this topic
    http://www.amazon.com/Economics-Iterative-Software-Development-paperback/dp/032189698X

    Reply
    • johanna

      Glen, I agree with you that we need to replan. However, if you focus on value, not cost, you might make different decisions.

      The value of a project (or a feature) and our ability to ship high-value often is what works. I happen to like agile for that. I also like staged delivery and design to schedule approaches. What they all have in common is that you can see the product come together. You can release “early” because you finish as you proceed.

      Software is about learning. We might learn what we want to do on our Italy trip. But Italy is not going to change before we get there. Some aspects of Italy might change, but unless there is a disaster, the land mass itself is not going to change.

      We can’t say that about software. The entire “what is valuable” conversation can change on a Tuesday morning. Working towards deliverables and reducing WIP work, so you can ship something. If those somethings are the most valuable, you win.

      I bought that book and will read it. Thank you.

      Reply
      • Glen B Alleman

        I’d suggest in the economics of software development, or the economics of any decision making process, value can’t be determined without knowing cost.

        ROI = (Value – Cost) / Cost

        Without cost you get a “divide by zero” error for the decision maker

        Reply
        • johanna

          Glen, you don’t need ROI to understand value. You can assess value in many other ways by comparing the relative value of projects. I explain this and have several examples in Manage Your Project Portfolio.

          I also have an article, Selecting a Ranking Method for Your Project Portfolio.

          In the book, I do say you might need to do a gross estimation, as I do here. But expecting an accurate estimate and basing corporate decisions around that? In my opinion, that is quite dangerous. You might end up like Sony in this post: Why Cost is the Wrong Question for Evaluating Projects in Your Project Portfolio.

          I have never seen an accurate ROI calculation. Never. Maybe some people can predict that much: how much you will sell and what the cost will be. You can manage to cost, especially once you are partway through the project/program. But I have been in way too many organizations where the sales were not realized. So the expected value was never what people thought it would be.

          Instead, if you say, “Who are the people who would find this valuable?” and you incrementally develop, you can manage your risk quite well. No estimates required.

          I am not against estimates. I estimate how long some of my work takes all the time. I am against basing project decisions on estimates. The cone of uncertainty is a subjective approximation of what Boehm thought happened in estimation. See Leprechauns of Software Engineering. The cone has no basis in reality.

          Since I have never seen a truthful ROI, and I don’t know how to create accurate estimates at the very beginning of the project (we have yet to become aware of some of the critical requirements), why should I believe in estimation?

          Now, that is for the software part of the project. I understand how to estimate–with pretty good accuracy–NREs. I understand how to timebox everything in a project or program. I understand how to create deliverables or work with the correct people to create deliverables they can deliver inside of 4 weeks or less.

          But using estimates as a surrogate for value? That leads to sunk cost and over-optimism.

          I am a huge fan of iterative planning and delivering. I have run projects like that since the 1980’s. My projects, where I was the senior engineer, and projects/programs where I was the project/program manager. I have had good results. I have never had good results when my clients insisted on deciding via estimate.

          I am only one person, so I can’t say estimation won’t work for some people. Clearly, estimates must work for you. But I have yet to be able to make estimates work for me, as a way of guiding project decisions.

          Deliverables work. Iterative planning works. Forcing us (a project team) to understand the requirements enough so we can deliver working product inside of 4 weeks and regularly after that–that works, too. I have managed small projects (a team of 3) and multi-million dollar programs, where the run rate included 20 feature teams, and had hardware costs. We had a ton of uncertainty, but deliverables showed us how to manage the risks. Not estimates.

          Reply
          • Glen B. Alleman

            As Ron Jeffries once suggested “the sky is a different color where you live.”
            In our domain, space, defense, and enterprise ERP, ROI is done with confidence for agile development projects all the time.

          • johanna

            Ron is one smart guy.

            I wonder if the learning is different in those projects. That you don’t expect to learn as you proceed (as much as in the projects I have experience with). Or, in the case of space and defense, you use the learning in one phase (or set of milestones) to inform the next set.

            I don’t think there is One Right Answer here. I do know that I have seen people re-estimate over and over to get the Politically Correct Answer, instead of delivering some chunks of working product. I bet you would agree with me that that kind of estimation is useless.

          • Glen B Alleman

            Good question. I’d suggest the learning is even more so, but at a higher level. Some of the those programs – invent new physics or have similar needs.
            What is stable (usually) are the needed capabilities. These are business or mission capabilities – “we need the capability to process provider enrollments for $0.17 a transaction, versus our current cost for our 7M insured clients. How we going to do that? Don’t have a clue yet, but need it by the 3rd quarter a year from now, when the SAP implementation is complete. And BTW you’ve got $180M to do this with. Don’t be late, don’t be over budget – within margins.

            There is no right answer except from a Principle Point of View – the 5 immutable principles of project success. From planting the garden in the spring to flying to Mars – page 5 of http://www.slideshare.net/galleman/principles-and-practices-of-performancebased-project-management

            These class of projects are usually “rolling wave” with increasing maturity of the deliverables. Here’s an example of “increasing maturity” in the insurance domain
            http://www.slideshare.net/galleman/project-maturity-flow-is-the-incremental-delivery-of-business-value

            Incremental capabilities at the macro level, are then developed using agile (Scrum), but the capabilities don’t “emerge.” The requirements and the technical design of how to deliver those capabilities “may” emerge for less mature items.

            But the senior managers of the business (health insurance) have a clear idea of what they want and are not subject to “we won’t know till we see it.” The agile parts are in the next level down from the Rolling Wave. In this example the Pilot and the Data come together to produce a demo of the conversion process and member reconciliation from the legacy system to the new system. And so one for each “increasing maturity” over the life of the project.

          • johanna

            Glen, I agree with your principles. In fact, you agree with me: use rolling wave deliverable-based planning.

            We might disagree: when do we use prototypes vs trying to deliver the real product? But, I bet we don’t disagree on that either.

            I don’t say, “let’s play and we’ll get it to you when we do.” That’s not risk management.

            I agree with you: use rolling wave deliverable-based planning to show people what we have completed. That’s the idea behind the agile roadmap and using target dates. Show people early and often what we have completed.

            I suspect the place where we disagree is the value of in-depth estimation. I have not found the use of in-depth estimation useful. I have found a gross estimate (a SWAG, if you will), immediately followed by deliverables and updating the plan (roadmap, so we are always working on the most valuable work), very useful.

          • Glen B Alleman

            Since there is a wide range to “projects” in a wide range of domains and similar governance processes – from self-governed to Federal Rules (HHS for example in Medicare disbursement systems), HIIPA, and others based on ISO 12207 (SDLC), it’s difficult to to assess the value of estimates in the absence of a domain.

            http://www.slideshare.net/galleman/paradigm-of-agile-project-management

            Is one view of this range and the applicability of not only estimating, but requirements, testing, IV&V, operations, and all other aspects of “writing software with other peoples money.”

            A SWAG immediately followed by deliverables may be the perfect approach in one domain, and legally forbidden in another.

            Domain first, then processes and practices.

  3. Pawel Brodzinski

    Hi Johanna,

    That’s a good read.

    I recently wrote an article around estimation in my context. I know it’s different than yours yet it may add to the discussion too: http://blog.lunarlogic.io/2015/project-management-estimation/

    The motivation was mostly selfish as I need to go through the whole argument almost always when I’m talking with our potential clients, yet I made the general case.

    Reply
    • johanna

      Hi Pawel, I like your article, too.

      I received feedback on the first draft of the program management book. I think I have to address this question: Given what is driving your project, what makes sense for your estimation?

      – If it’s schedule, you work to a target, and keep showing progress so you build trust.
      – If it’s feature set, you want to produce features as quickly as possible and show progress. You build trust and can change the features. Teams might have to estimate Your Minimum Indispensable Feature set at times during the project.
      – If it’s low defects, you might want to consider working towards targets and keep producing to show progress and build trust. You need release criteria. You might need landing zones to get closer and closer to the “ideal” release.
      – If it’s cost, get a cost target, produce small increments all the time, build trust, and keep evaluating if it’s worth doing. (This is really difficult with hardware, but not impossible.)
      – If it’s optimizing people and their capabilities (how people learn, not optimizing for each person’s utilization), I’m not sure what to suggest yet. I’ve only been on one project where we had the project to keep people from leaving.
      – I don’t know any project where the project environment drives the project. I don’t know what to suggest if process, rather than product is king.

      For why I chose these, see my Estimating the Unknown Series.

      As you and Glen have pointed out, estimation does depend on your context. It depends on your domain. One solution is not going to fit all.

      Reply
  4. Patrick Edward

    Hey
    Interesting read.
    My issue however is that I can’t get any project funding without showing a list of Prioritised Programmes with quantifiable benefits.
    Have you had experience of that situation, or if you did, how would you Plan/Estimate for it given the inherent uncertainty of software development like you mention.

    Reply
    • johanna

      Hi Patrick, ah, the managers want to manage the project portfolio by estimation instead of value. That’s not a good idea. See Cost, Value & Investment: How Much Will This Project Cost? Part 1.

      Take a look at Why Cost is the Wrong Question for Evaluating Projects in Your Project Portfolio, to understand part of the value question.

      The real issue is this: When you have to decide among projects, you need to assess the relative value of doing each project. You will have three kinds of projects: keep-the-lights-on projects; projects that move your products along a predictive path; and transformative projects.

      You can estimate the first two kinds of projects. (I have the estimation series, and Essays on Estimation to explain how to do a gross estimate at the beginning.)

      However, you cannot easily estimate the transformative projects. Those are the projects that make the money for your organization and catapult you to another place.

      Those projects are the ones where you need to have interim milestones, replan, and re-estimate at those interim milestones. I like to use agile or staged delivery, because you see features emerge, not documents.

      To answer your question: The benefits you show are the value from doing the project/program. What is the purpose? What are the success and release criteria? If you don’t know those, maybe you charter the project anyway to articulate them. (See Manage It! Your Guide to Modern, Pragmatic Project Management.)

      But the idea of an accurate crystal ball? Not if you’re doing something new. You have a small idea. That estimation is not going to be accurate.

      Reply

Trackbacks/Pingbacks

  1. Five Blogs – 26 January 2015 | 5blogs - […] We Need Planning; Do We Need Estimation? Written by: Johanna Rothman […]
  2. Johanna Rothman : What Development & Test Managers do in Agile Organizations | Project Management Buzz - […] « We Need Planning; Do We Need Estimation? […]
  3. Johanna Rothman : What Model Do Your Estimates Follow? | Project Management Buzz - […] you have? If you have large requirements, follow my advice and use the percentage confidence, as in We Need…
  4. How to Create a Game Development Project Plan | GBGames - Thoughts on Indie Game Development - […] We Need Planning; Do We Need Estimation? by Johanna Rothman: A quick read about the idea that detailed estimates…

Submit a Comment

Your email address will not be published. Required fields are marked *