Time for Innovation in Timeboxes?

As part of some recent consulting and training, one of the project managers asked, "How do you make time for innovation in timeboxes? If everyone's busy all the time, how can you allow people time to think for real innovation?" Good question. I asked how people had time for innovation now. The PM wasn't sure, but he was sure it happened.

I do agree that you can't tell people, "It's Wednesday at 10:28am. Be innovative." That doesn't work. Innovation occurs when people connect ideas that weren't connected before. There's a substantial thinking component before people can see those connections.

What I don't understand is why the PM thought timeboxing makes innovation less likely to happen. Sure, people focus more on the work at hand, but for me, that makes it more likely that I'll see connections as I finish one thing or when I move to something else. I've been able to finish work, which frees me up to do something else, and to let the thinking part of my brain go to work.

If I have a bright idea, I can always put it on the backlog to address at some other time. (This is the idea behind fieldstones in writing.) I haven't forgotten it, and I have plenty of time to let my subconscious go to work on it. When I don't timebox, I'm more likely to distract myself and try to innovate without the necessary thinking time first. That's why I timebox almost everything I do :-)

I'm not sure the PM bought this--but this is what happens for me. What about you?

Letting Go of BDUF

I've taught several workshops where people wanted to learn how to start adopting some agile approaches. They knew about timeboxing, but didn't quite see how to make it work. The part they were missing was having working valuable product at the end of each timebox.

I explain that to the participants, and they nod sagely. Then I ask them to do a simulation. Practicing in a workshop is a safe place to practice. If they don't do something quite right it's still ok. Normally this part proceeds well. Some folks have a little trouble, but most people start changing how they work by the third iteration.

At a recent workshop, things didn't go so well. One group took five iterations to complete the first part of the project when I'd expected three iterations (when designing the simulation). But take a look at a chart of their first project's velocity.

They couldn't predict what they could finish--or not--in an iteration. One of the reasons they had so much trouble is that they did not implement by feature. They attempted to build the architecture first. But as soon as they started attempting features, the architecture didn't quite work. Instead of throwing away a not-quite-right architecture at the end of iteration 2, they decided to stick with it, because of all the time they'd invested.

They had a different experience in the second project. In their first iteration, they decided to do an architectural prototype, but not deliver anything. Ok, I could live with that. They had a working architecture, and if they'd put things together, they could have been halfway done with the project. They chose to leave the first iteration as a prototype.

In planning for the second iteration, they told me (senior management), they were going to work on the prototype more. I asked what they would have to show me. It wasn't going to be product, so I told them that was unacceptable. They didn't plan much for that iteration, but delivered plenty. Same thing with the next two iterations.

They never did have a predictable velocity, because they were caught in the trap of pre-planned architecture instead of seeing the architecture evolve as the product grew. I asked them if this ever happened in their organizations. Predictably, it did.

This may be the most difficult idea to help people see--that they make more progress implementing the most valuable features one at a time, and letting the architecture evolve. Big Design Up Front (BDUF) slows down the project. You can't predict what the architecture needs to be.

Timeboxes, Iterations, and Orthodoxy

If you haven't read Duane Nathaniel's thoughtful comment on What Happens When You Can't Finish What You Wanted in an Iteration?, do so now. Duane makes some great points.

RUP has iterations; they're not timeboxed--they're deliverable-based. (Take a look at the link that Duane points to.)

In the RUP, an iteration results in a deliverable. If it takes a little longer to get to the deliverable, that triggers some decision points. But while the RUP uses iterations, it doesn't use timeboxes.

Timeboxes are time-based iterations, not deliverable-based. Which is why if you're using timeboxes, it does matter if you keep to the timebox, because the timebox allows you to compare what you completed in each timebox.

This says to me: decide if you want a deliverable-based iteration or a timeboxed iteration. The two are different. What you get is different. With timeboxes, it's much easier to compare velocity and track estimates. With deliverable-based iterations, it's easier to deliver some specific thing to someone else, and create decision dates in the schedule. Your project might need one or the other. Maybe pieces of the project need both.

Duane was right when he said project managers need to think about the tools they use, and make sure they've chosen the right one for the situation.

Estimating What's Remaining to Finish

Pawel caught me being ambiguous. See his comment, "1. I've seen features/fixes which required 2 days to be developed and released."

Sorry, me too. But what I tried to say was this: A feature was estimated to be some duration of person-hours. Those person-hours have come and gone. The feature still requires another 10-12 person-hours, according to the developer. In that case, I don't trust the estimate of 10-12 person-hours left. In my experience, the reasons for the delay generally have not been addressed. If interruptions caused the delay, more interruptions could still happen. If the delay was caused by having to change the architecture, that could happen again (and I suspect it's likely). In my experience, until the feature is done, it's quite difficult to predict when it will be done, which is why I like to start a new iteration to make sure it gets done.

I tried to imagine what would convince me that an updated estimate was more accurate. In the past I've only had one thing that would convince me: I would have to see visible progress of more work complete (more pieces of the feature complete) to agree that the estimate of what's remaining to finish is correct. I'm trying to think of more things, and can't right now.

What Happens When You Can't Finish What You Wanted in an Iteration?

I ran a little workshop today about transitioning to agile. I was talking about timeboxed iterations, and one of the participants asked this question. "So I don't quite finish one of the features I want to finish in this iteration--and it's the end of the project. I think it's going to take me a couple more days. Do I extend the iteration or do another iteration?"

I said, "End the iteration at the end of the timebox." There are lots of good reasons to do so: retain the reasonable pace and rhythm of the project, be able to gather data to compare this iteration to previous iterations, and to see why the team thought they could finish all these features in an iteration but couldn't.

The participant didn't agree with me. He has deliverables to other people. He can release at the end of an iteration. If he starts another iteration, why should people wait 4 more weeks rather than 2 days for the release?

Aside: I've never seen a 2-day estimate at the end of the release actually finish in 2 days. I've always seen it take several weeks. But let's assume it really is just 2 more days of work, and feature is the only thing left to do. He could start another timebox and end it early. Or, why not just use an interim build as the deliverable, if the build is not for an end customer?

The class had a vigorous discussion about whether to end or continue the timebox. I hope that he tries ending the timebox when the time is over--otherwise it's not a timebox. It's too easy to let this iteration go over by a couple of days, let the next one go over by a week, and pretty soon you've got 5-week, 6-week, 7-week iterations, and the value of being able to do iteration re-ranking of features and the ability to make releasable software every x (here 4) weeks is gone. You've got something that looks like a staged delivery lifecycle, assuming you're integrating as you go.

So, what do you do if you have just a little bit left in an iteration? I suggested that aside from ending this timebox on time, that they reduce the timebox duration to two weeks, and see if that improves their estimation and iteration planning. (My concerned participant was not excited about that option.) If you have experience with this, please comment.