Management Debt, Technical Debt, and Decision-Making

Dave and Bob have great comments on my post, Might Three Backlogs Be Better Than One?. Dave is describing situations where management is making reasonable decisions, not incurring management debt, and by extension, technical debt. Bob and I have experience with significant management debt. (Take a look at Musings About Management Debt for more information about what I mean by management debt.)

Let me provide a little more perspective on these situations. The clients range from large systems of up to a million lines of code (but still software-in-the-small) to software-in-the-large systems of 14, 15, 16 million lines of code. These systems have been in existence for more than 5 or 6 years. They are the flagship, and sometimes only, product that creates revenue. In each case, management is fairly new. The product owner players are new. Many of the technical people are new.

No one knows enough to make good decisions. Yes, they should have one backlog, and they don't know how to prune the defect list. When you're talking about more than a few thousand defects, the number is just too big. When your builds take longer than one day, when you have no automated tests, when you're using a configuration management system from the '70s (ok, not quite that bad, but close), and you don't know how to break down the work into small chunks, it's all overwhelming. Oh, and don't forget the customers breathing down your neck for not just the defects you need to fix yesterday, but the features the sales people promised tomorrow.

One VP told me he could not make the decisions. He was concerned about who he needed (“do I need all of Engineering??”) and the time to rank into one backlog (“We'll be here all day and we won't be able to trade off against features and defects”).

He's inherited a situation not of his making. He knows that if he can get people to start working in an agile way, they could make progress and finish some work. That would make their decision-making easier. But he has no roadmaps for feature sets for the product to see into the short-term or long-term future. He doesn't know how many of the 1000's of defects are still valid. (He thinks more than he wants.) Everybody is nervous. No one feels as if he or she can make a decision alone. The notion of a product owner as a “single, wringable neck” (do I credit Mike Dwyer or someone else with that phrase?) is not something they are ready for. They wanted a committee. I did manage to talk them into a committee of 2: one technical person and one marketing person. That technical person is the erstwhile project manager and the marketing person is talking himself into being a product owner.

I don't claim these folks are agile. I don't think they would either. They are working in timeboxed iterations. They are trying to finish valuable chunks of work in those timeboxes. They are struggling with how to make their decisions.

One thing I know about decisions is that the smaller the decision, the easier it is to make it. When you have to make tons of decisions that last forever, it's really difficult, especially if you think you don't know enough. At first, they all thought their technical debt was too huge to make inroads. But when I explained how to break down their technical debt into user stories, and that they didn't have to tackle everything at once, that if they could just get the build to take less than a day they would already be ahead of the game. They didn't need to stop developing features or fixing problems to address their technical debt.

As we see more people who have used a phased approach to development for years, and have not had tremendous success move to agile approaches, we will see people who may not be ready for real agile, but are ready (desperate?) to try something else that will provide value. We need to help them move from their previous insufficient decision-making to decision-making that will work. I do like baby steps. (See  Small Steps Are Good; Be Careful What You Call Those Steps).

I do worry about the issue of three backlogs. In the face of no decisions, I still think it's better to make some decisions even if the outcome of those decisions is not one backog. Maybe other people have had better results nudging decision-free organizations to decisions faster than I have. I do think that the visibility of the different kinds of work can help people out of their defeatist attitude of “We have so much to do, we can never finish it.” One of these clients is actually pruning their defect list this week. The reason they can do that is because they took one story off the technical debt list, finished it, made so much more progress in just two weeks that the person-who-might-be-called-the-product-owner-but-is-resisting-the-role asked if they could finish a few defects in the next iteration, and how many did they really have anyway? They have bought themselves room to breathe.

Agile–real agile–is about making decisions based on business value, and delivering valuable chunks of work in a small timebox. If you have an organization that can't decide on business value, you are in trouble. If you have better ideas, I'd love to hear them.

7 Replies to “Management Debt, Technical Debt, and Decision-Making”

  1. Johanna,

    With all due respect, the way you describe the approach to the situation in this post really sounds reactionary – scrambling to deal with a mountain of tactical issues all falling on your heads at once. I’ve been in similar situations and I can appreciate the difficulties, especially the difficulty of taking a deep breath and a step back to try and see a way out of the status quo. It can be very tough, but it need not be insurmountable.

    Sometimes one of the hardest things for client management to understand is that the only way to get off the treadmill is to just get the hell off the treadmill. As long as they’re running at full speed to try and cope with everything as fast as possible, they will never be able to fix the root causes of their problems. If there really are, literally, /thousands/ of reported defects, then it is a near-certainty that most of them can safely be ignored. Otherwise, the applications wouldn’t be viable at all. Really. Been there, done that. The trick is to recognize which defects to ignore. AFAIK no one can do that while they’re sprinting as fast as they can, hearts pounding, out of breath, and going nowhere because they’re on a treadmill.

    Quite often, the attitude of “We have so much to do, we can never finish it” happens when people believe they have more to do than they really have to do. It’s just that no one quite knows how to tell which things they have to do and which things they don’t.

    Something that may or may not help could be to broaden the range of alternatives you’re considering. I’m reading between the lines here, and please forgive me if that leads me to faulty assumptions, but I’m going to stick my neck out and toss out a couple of ideas.

    You’ve used the terms “product owner” and “timebox,” and also the term “real agile.”

    Agile doesn’t say anything about timeboxes. Count the number of times the word “iteration” appears in the Manifesto. It’s a nice, round number. Until recently, all the popular agile methodologies have been based on timeboxed iterations, but “agile” as such doesn’t call for that, necessarily. The key thing we’re after is to deliver value incrementally and regularly. The mechanism for that can be, but doesn’t have to be, timeboxes. Maybe in the case you’re describing, a timebox-based approach isn’t a good fit.

    The role “product owner” is defined by the process framework, Scrum. It is not a generic “agile” role. You don’t need a “product owner” to be doing “real agile.” In a situation such as you describe, who in their right mind is going to volunteer to be the “single wringable neck?” Not I. Not you, either, I’ll wager. David Schmaltz’s concept of the “project community” may offer value in that situation. It brings a spirit of collaboration and a sense of collective ownership that are very consistent with agile principles, and often more practical than the “product owner” role.

    Besides that, you may find that lean thinking offers a gradual approach to change that may be easier for an organization in the condition you describe to consume and act upon than textbook “agile.” It starts with getting to a clear definition of “value,” then using Theory of Constraints tools to improve continuous flow, and thirdly removing waste from the process when and as feasible. Agile is really about software development; lean is about end-to-end process improvement, which sounds more like the nature of the problems you’re coping with.

    The software development piece is…well, it’s just a piece. It’s not the whole problem. You’re talking about ongoing maintenance and support activities, not discrete software development “projects.” If the place is like most IT organizations, they’re spending about 80% of their time and effort on ongoing maintenance and support rather than discrete development projects. That may be one of the reasons it’s been hard to get the “product owner” role up and running smoothly, and hard to know which items to include in each timeboxed iteration. The iterative model doesn’t quite fit the need…if my between-the-lines reading is accurate, that is.

    If this is a current situation and if you think a fresh pair of eyes might add value, please contact me off list about it. As bizarre as it may sound, I love situations like that.

    Amber,

    I don’t know you but I like the way you think, I think.

    “…after awhile, we stopped doing that, in favor of just fixing something right then and there.”

    I recognize this as a generally-accepted good practice known as “fixing broken windows.” Good for you!

    Cheers,
    Dave

  2. My experience with the label of ‘Bug/Defect’ and ‘New Feature’ has been that it is highly charged, mostly semantic and ultimately a useless debate. So that’s why I would not want to split backlogs based on these labels. At the end of the day, what is the benefit if doing X story points of defects, if the customers don’t really care about them (ie. low priority) but are really clamoring for some new feature (ie. high priority). Splitting the backlog distract from the primary purpose of adding value.

    Then we get to the ‘Technical Debt’ label. As people had said in comments to the afformentioned post, all user stories need to be tied into a specific and -visible- benefit to the user. I have tried putting technical debt as user stories and it has always been useless. Sure it would get done, and sure it would explain the X amount of story hours spent. But it was more of a placeholder. It wasn’t tested in the same way. It wasn’t “released” as a benefit to the users. The project owner had no way of telling priorities.

    As for a solution, I think that agile empowers customers in a way that tradiaional processes doesn’t and that’s part of what’s so great about it. But I think that developer’s do a disservice when we relinquish too much power. The entire point is to let them manipulate priorities on -features- they want, NOT technical impementations.

    So assuming you never make a user story for a technical debt – how will it ever get fixed? It will get fixed whenever the developers get so fed up with it that they change it. Yes, that does require having good developers and it also requires putting trust in them. But how is it better to trust a -technical- debt in the hands of a non-technical person?

    So as not to -throw away- the list of technical debt, one thing we used to do was comment //TODO: blah when we noticed a technical debt. But after awhile, we stopped doing that, in favor of just fixing something right then and there. Turns out being able to list a technical debt in a backlog or a comment is a crutch that prevents us from just fixing it in the first place.

    What will be the effect of doing this, particularly for projects with large pieces of technical debt? A very unsteady velocity. One week, they may get X story points and they another week way way less, since they’re spending their time on a piece of technical debt. Normally that’s a bad thing, but the technical debt IS a bad thing, so it’s appropriate. Once more of the big pieces of debt is gone, the velocity will stabilize. Padding the velocity with technical debt stories just artificially stabilizes the velocity which is not helpful for estimation.

  3. “Turns out being able to list a technical debt in a backlog or a comment is a crutch that prevents us from just fixing it in the first place.”

    Yes, one of the dangers of processes is they become a shelter for people to hide behind – as in “I did it the way the book said I should and it didn’t work – not my fault guv”. Sometimes just fixing something is beneficial, and can buy some breathing space.

    I also agree with dave “The role “product owner” is defined by the process framework, Scrum. It is not a generic “agile” role. You don’t need a “product owner” to be doing “real agile.”” By getting hung up on a concept of product owner or any other term, you are losing sight of agile, you are replacing it with another formal process driven method and hiding behind the terminology “sorry mate, no product owner, no agile and hence no product”.

  4. I have started to think there is a difference for many of us in the “agile” world between Agile and lean projects. Where by the lean has a little more formality, specialists, heavier processes, etc., and therefore might have more debt and heavier decission making. I release many Agile people might argue with me, but, for example, in lifecritical complex systems being a lot more careful, formal, specialize, and even in some cases slower (backlog) is often in order.

  5. Pingback: yentit.com
  6. Pingback: Anonymous
  7. Pingback: www.webbiru.com

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.