Failed Fast at Agile 2011, Learned a Lot

I prepare for my speaking and workshop engagements. This year, I've been all over the world. I've had a great time, and my clients and audiences have had a great time, too.

Well, except for this past week at my session, “The Budgeting Black Hole: Predicting the Unpredictable” at Agile 2011. There, I bombed. I didn't just do somewhat badly, I was a total disaster. When I say disaster, I mean disaster. I am accustomed to getting mostly high marks on my feedback forms, with some very low marks because I say provocative things that some people don't like. That's fine. Here, I got zero high marks. Zero. One medium mark, and that was generous. All the rest were low and very low. So when I say I bombed, I mean it. Disaster.

Here's what I hoped would happen in the simulation:

You would try to predict something. How can you? It's a new project. You can't predict anything. You need experience. So you do one iteration. Based on your iteration/experience, you choose what to do with the project portfolio, including the budget. Now you do one more iteration, again assessing the progress you made. You make another prediction, based on your now two iterations worth of experience and your ability to re-jigger the project portfolio. Do the final iteration, count the money and see where you stand. Since we had multiple companies, we had a way to compare. You don’t get this at work. It was a laboratory experiment. Oh well. It’s too bad the simulation failed.

For many reasons, the simulation failed after one iteration. The only good thing is that I recognized that fact and did not try to continue.

And, I learned a lot, about simulations for managers and the Agile conference. Sometimes, I am not too bright. I am always the one talking about how busy managers are, how they don't have the time to waste. Dope-slap on me. These people wanted answers. If I'd explained at the beginning that I was hoping to do an experiment about the ways they might have managed their budgets, they might have said, “Uh, heck no,” or “Well, we'll try it,” or something else, but I didn't give them a chance. Nope, I just assumed they would want to experiment with me, because iterative and incremental budgeting is new, and no one has really figured it out. But I didn't write the session description that way. Nope, I wrote the description so that the session would get accepted.

So here's what I did so that these people who spent any time with me would feel as if they got their money's worth:

  1. I uploaded a substantial pdf with everything I know about incremental budgeting. Feel free to download it yourself. It's at The Budgeting Black Hole: Predicting the Unpredictable. In the upper right corner, where it says “Download session PDF”
  2. I started a LinkedIn group, Agile Project and program portfolio management. Want to join? I'm moderating the group for now. If you're an agilist, interested in project management, program management, curious, please ask, and I will add you.
  3. I have business cards from some of the people in my session, so I can send them the pdf directly. If you know of any of the people in my session, and they said to you, “Is Johanna an idiot?” you can let them know that I'm not most of the time. Every so often I am, and when I am, it's really a doozy.

I failed fast. I learned a lot. I would have preferred to fail in not quite so public an arena. Oh well. I decided the fastest path to putting this behind me would be to admit it (transparency) and to write about it (the blog post and the pdf), and continue from there. So that's what I'm doing. Let me know what you think.

14 thoughts on “Failed Fast at Agile 2011, Learned a Lot”

  1. In your defense, your session description DOES say it’s a simulation! I’m curious in what way it failed? You say many reasons. Were they able to accurately predict, which wouldn’t happen in the real world?

    Anyway – thank you for sharing this, it makes me feel better about all the times I’ve screwed up a tutorial. If Johanna can fail, anyone can! And who says we don’t learn from failure? Clearly you have a good action plan to improve as a result of this.

  2. Congratulations on recognizing a failure and bailing on it as quickly as possible. Not enough of us in the Agile world follow our own advice to fail fast!

    One other comment about this quote: “But I didn’t write the session description that way. Nope, I wrote the description so that the session would get accepted.”

    Why does that seem so wrong to me?


  3. I would be interested in seeing some of the reasons why you feel the simulation failed. I re-read your session description and I can infer that it was a simulation (it was classified as a tutorial though) and I can understand why some people may not have got that. I’ve heard the Agile 20XX crowds can be tough when it comes to experiential learning.

    Thank you for writing this post, you have a great deal of admirers and have earned a tremendous amount of respect within the Agile community. Sometimes things don’t go as planned and I think many people can learn a lot simply by how you responded to this problem!

  4. Jason (et al),

    Over the years since my first ADC2004, I have noticed that experiential learning sessions seem increasingly difficult to satisfy people with. Perhaps more people are coming looking for “answers” rather than raising questions to guide them to answers that fit their situations. I remember the early ADC/AGILE confs and how great those experiential sessions were and the overall Open Space sense of it all. Since then, I’ve sat in such sessions and found an increasing number of people less than enthusiastic with this use of the time compared to the desire for bulleted viewgraphs of how-tos. Perhaps companies are not that interested in paying $1600+ for “experiences” compared to “answers” and semi-free consulting.

  5. hi Johanna,
    Did you do a tryout?

    I personally think that for simulations you need at least 3 versions before it becomes any good. (I think that is the same for normal sessions, but good presentors can hide more with a new session.)
    I created a few games. Every time I think, now we got it right the first time. We never did.

    And with simulations I think it is even more important to pair-coach/pair-present.
    That way when you bomb, you probably realize while you are doing (like you did) and you have a sparring partner where you can find idea’s.
    (Or that encourages you to keep going as she sees a outcome of your sessions you missed.)

    I hope you can get in tough with a few of these people and see what they want from it.

    Last year when I did our ‘burning house’ session at a local conference, some people did not like the fact it was an interactive session either. They wanted to be told the outcome….

    I also think that the Agile 2011 organizers are part to blame. What I have noticed is that the conference organizers (or at least the people giving feebback) seem to prefer new sessions over session that have been proven to work at other conferences.
    For me Agile 2011 should be the best of agile around the world since last year. In reality it are mostly new sessions that are then replied around the world.
    Which basically means that most sessions are a try out at agile 2011. And that is a big shame as you have proven that even very experienced presentors sometime screw up.
    (And that is OK)


  6. Johanna,

    It’s very brave of you to talk about this on your blog. When I screw up, especially publicly, I tend to spend so much time sulking that I can’t get myself to retrospect and blog about it. Posts like this make the blogosphere real and personal.

    Nice job.

  7. Jorgen Hesselberg

    Seems to me you did exactly the right thing, Johanna. Some things don’t work out the way we intend – especially if you’re being innovative or taking some risks, like you were.

    Love the way you responded; instead of being defensive and arguing that ‘the audience just didn’t get it’, you accepted responsibility and took meaningful actions to help alleviate the situation.

    Kudos, Johanna – we can all learn from this!


  8. Johanna,
    I admire people who can fail well, people who are graceful about it and use it to both learn from and teach others. All the better when you fail early. I liked how you noticed that you, “wrote the description so that the session would get accepted.” And I like how you made further resources available. Even the LinkedIn page is open to read without joining. Nice touch.

    But I too think this post would have been much more valuable had you detailed the way in which you failed. Show rather than tell? We don’t even get to weigh in on whether it qualifies as a failure.

    I once failed at an doing an alignment exercise that had worked great last time I delivered it. Using a laser pointer, I asked people to point their chairs toward the dot on the wall, as I clicked it on and off, and occasionally switched walls. Easy to see how we fall into a pattern of which wall we want to face, without noticing the other walls that may get the light now and then. First time it was obvious. Second time, just confusing. I realized the wording and delivery were critical, and slightly different. Were I to do it again, I’d practice in front of a few groups first and refine the pitch.

    1. Steve and Matthias,

      Here are the ways in which I think I failed:

      1. I did not set their expectations at the beginning.

      2. I had people spend 10 minutes attempting to estimate how long the initial projects might take. They had their choice of at least three projects. The reality is they could have spent three hours and still not have known. I could have shortened that timebox to 5 minutes and still gotten the point across quite nicely. In fact, if I had, they might not have been quite so frustrated going into the simulation. (Well, they’d already started the simulation, but they didn’t realize it.)

      3. I had the project work set up as 3 4-minute days with 3-minute standups in between the days, with a final 3-minute demo. I did not emphasize the need to coordinate between the two teams at the tables at the standups. These were managers, who are not familiar with how the technical teams work at the team level. Some of them were, but not all. I bet some of these people felt as if I’d tricked them, which was not my intent. I could have set the simulation up as one 4-minute day as an iteration, and I could have said, “Your job is to build the best product you can in your timebox with your team.” Many of these folks are not familiar with the technical practices, so they did not practice continuous integration or pairing, so their products were not what I expected. Again, I bet they felt set up to fail, which was not my intention. In a simulation, if the people feel set up to fail, the simulation designer has failed.

      4. If I had shortened the estimation time and had the team managers reevaluate the project portfolio after each “day”, we would have had the result that I wanted, which was frequent project portfolio evaluation, which would then have led to incremental budgeting, which would have led to a discussion/simulation of how to do incremental budgeting where you work, which was the objective.

      I don’t want to blame the people in my session for being human. I asked them to work in ways some of them were not even aware.

      At the conference, the sessions are categorized by beginning, practicing, and advanced. No one looks at those, I’m sure when they choose sessions. They look at the titles. (Besides, if you have eyes older than 25, you can’t see those designations on the daily schedule.) And, even if they look at those designations, everyone thinks they have been practicing agile. Even if they have been practicing not as effectively, or even ineffectively. Wait until you read my post on organizations with no customers/no product owners!

      That’s why on the first page of the handout I have a “here’s what agile means” picture. And, we didn’t have enough handouts when we started, because I was supposed to estimate the number of people for the session. I thought, “how many could there possibly be?” and estimated 50. Over 100 people showed up. And, because I’d stood in line for almost 40 minutes to get lunch, I only arrived at the session with 2 minutes to spare. When I saw that many people, I asked the volunteers to get more copies made. They did, and returned with copies, but they weren’t stapled. I asked the volunteers to staple them. That took until we were into the estimation. By that time, I didn’t want to interrupt where we were to pass out a handout that was now meaningless in the scheme of things.

      Can you say “a series of unfortunate events” ?

      I am the professional. I am supposed to make sure that things go well. I’m trying to remember what my trainer at the gym says. (Aside from saying he’s super-strong 🙂 I’m not Superman. On the other hand, in most circumstances, I should be able to give most people enough of what they came for, so they leave relatively happy. I did not here. The participants weren’t the only ones who were disappointed; I was quite disappointed!

      Hindsight is much clearer. I have run this before, and had better results. Certainly, many of these people wanted “the answers.” I’m not surprised as I think about the audience. They are managers! Why wouldn’t they? But I did not do a number of simplifications I know I could have done that would have made things proceed better. That’s why I think I failed.

  9. I had the privilege of sitting next to Johanna on the flight from Salt Lake City to Atlanta on the way home. We commiserated about how the audience rated our presentations..before succumbing to the fatigue of a long week. I empathize with Johanna’s feelings. She actually made me feel much better because I only had two or three folks (out of 58) hammer my session. I took a different approach to this year’s conference. I flat out stated that my quick session would be a lot of knowledge sharing with limited interaction/experiential learning. A lot of people liked it – food for thought, tips, lessons learned. The folks that didn’t wanted ‘an experience.’ As I reflect on our conversation and this post, the key takeaway is fail fast, learn fast, forgive yourself and move on! Agile2012 is less than a year away! Plenty of time to apply what we learned! Hang tough!

  10. Hi Johanna,

    I am very impressed with your openness!

    I was in your session and I have to admit, I was very confused. Even though I am not a manager and I am quite familiar with Agile practices… I found myself making flower baskets, all the time wondering what the purpose was. Due to the unexpected high number of participants, we also run out of flower-building materials after the first Sprint, so it was impossible to go through a second Sprint. More Sprints would have allowed us to understand the simulation goal, and actually learn from the simulation.

    I also create and organise simulation workshops and games. It’s not easy. What I’ve learned from your session, once again, is that these simulations need to be very simple. The facilitator needs to explain the purpose and the rules step by step. It seems a bit silly to do this for a group of smart people, but it turns out to be necessary to avoid lots of confusion and to get a game going and finished in 90 – 180 minutes.

    I have also learned that the number of people that can participate in such a session should be limited. One faciliatator can only help one table, maybe two. I think we had 6 tables? I’ve seen sessions at Agile2011 where the volunteers closed the door when the maximum number of attendants had been reached. It’s not nice to disappoint the people who can’t get in, but it saves the session for those who did.

    But my greatest learning came today, when I read your blog. You must have so much courage to blog about this! I hope one day I will be able follow in your steps and be so open about the things that went wrong.

    Thank you for setting this example,

  11. I did something similar with similar results. I licked my wounds and mentally hid in the closet for a day or two. Last week I got a request from what must have been the one person who thought it was worthwhile to do a similar presentation for another group. I will apply what I have learned and push forward. This time it will be better!

Leave a Comment

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: