How 100% Utilization Got Started

Back in the early days of computing, machines were orders of magnitude more expensive than programmers. In the ’70s, when I started, companies could pay highly experienced programmers about $50,000 per year. You could pay those of us just out of school less than $15,000 per year, and we thought we were making huge sums of money. (We were.) In contrast, companies either rented machines for many multiples of tens of thousands of dollars per year or bought them for millions. You can see that the scales of salaries to machine cost are not even close to equivalent.

When computers were that expensive, we utilized every second of machine time. We signed up for computer time. We desk-checked our work. We held design reviews and code reviews. We received minutes—yes, our jobs were often restricted to a minute of CPU time—of computer time. If you wanted more time, you signed up for after-hours time, such as 2am-4am.

Realize that computer time was not the only expensive part of computing. Memory was expensive. Back in these old days, we has 256 bytes of memory and programmed in assembly language code. We had one page of code. If you had a routine that was longer than one page, you branched at the end of a page to another page that had room that you had to swap in. (Yes, often by hand. And, no, I am not nostalgic for the old days at all!)

Minicomputers helped bring the money scales of pay and computer price closer in the late ’70s and the 80′s. But it wasn’t until minicomputers really came down in price and PCs started to dominate the market that the price of a developer became so much more expensive than the price of a computer. By then, many people thought it was cheaper for a developer to spend time one-on-one with the computer, not in design reviews or in code reviews, or discussing the architecture with others.

In the ’90s, even as the prices of computers, disks, and memory fell, and as programmers and testers became more expensive, it was clear to some of us that developing product was more collaborative than just a developer one with his or her computer. That’s why the SEI gained such traction during the ’90s. Not because people liked heavy-weight processes. But because especially with a serial lifecycle, you had to do something to make system development more successful. And, many managers were stuck in 100% utilization thinking. Remember, it hadn’t been that long before when 100% utilization meant something significant.

Now, let’s go back to what it means when a computer is fully utilized and it’s a single-process machine. It’s only a problem if the program is either I/O bound, memory-bound, or CPU bound. That is, if the program can’t get data in or out fast enough, if the program has to swap data or program in or out, or if the CPU can’t respond to other interrupts, such as to read the next card from the card reader. If it’s a single-user machine, running one program, maybe you can make allowances for this.

But if it’s a multi-process machine, if a computer is fully utilized, you have thrashing, and a potential of gridlock. Think of a highway at rush hour, with no one moving. That’s a highway at 100% utilization. We don’t want highways at 100% utilization. We don’t want computers at 100% utilization either. If your computer gets to about 50-75% utilization, it feels locked up.

Why 100% Utilization Doesn’t Work for People

Now, think of a person.When we are at 100% utilization, we have no slack time at all. We run from one task or interrupt to another, not thinking. There are at least two things wrong with this picture: the inevitable multitasking and the not thinking.

We don’t actually multitask at all. We fast-switch. And, we are not like computers, who, when they switch, write a perfect copy of what’s in memory to disk, and are able to read that back in again when it’s time to swap that back in. Nope, because we are human, we are unable to perfectly write out what’s in our memory, and we imperfectly swap back in. So, there is a context switch cost in the swapping, because we have to remember what we were thinking of when we swapped out. And that takes time.

So, there is a context switch in the time it takes us to swap out and swap back in. All of that time and imperfection adds up.

Now, let me address the not-thinking part of 100% utilization. What if you want people to consider working in a new way? If you have them working at 100% utilization, will they? Not on your life. They can’t consider it. They have no time. (For why, see the practice and integration part of the change mode in Change is Inevitable.)

So you get people performing their jobs by rote, servicing their interrupts in the best way they know how, doing as little as possible, doing enough to get by. They are not thinking of ways to improve. They are not thinking ways to help others. They are not thinking of ways to innovate. They are thinking, “How the heck can I get out from under this mountain of work?” It’s a horrible environment.

When you ask people to work at 100% utilization, you get much less work out of them than when you plan for them to work a roughly 6 hours of technical work day. People need time to read email, go to the occasional meeting, take bio breaks, have spirited discussions about the architecture or the coffee or something else. We seem to need spirited discussions in this industry! But if you plan for a good chunk of work in the morning and a couple of good chunks of work in the afternoon and keep the meetings to a minimum, technical people have done their fair share of work.

If you work in a meeting-happy organization, you can’t plan on 6 hours of technical work. You have to plan on less. You’re wasting people’s time with meetings. Hmm, maybe that should be the subject of my next lightning talk.

Acknowledgments:

I thank Yves Hanoulle, Pete Miller, Don Cox, Rich, Markus Gärtner, Lisa Crispin. I hope I didn’t miss anyone. I can no longer find the twitter feed. Pete and Rich, I promise to write more about technical debt in the future.

I’m trying to describe the costs of multitasking. Here’s what I’ve got so far:

There are three parts to multitasking:

• Stopping the work you’re doing. The stopping cost is the time it takes to mark your place, save your work, etc. You haven’t stopped thinking about what you’re doing, but when you stop to take a phone call or answer a question, there’s a stopping cost. If you’re in flow, this is surprisingly high.
• Swapping out what you’re working on. The swapping out is the act of clearing your mind of the work you’d been doing so you have room to swap in the new work. If you were in flow or concentrating deeply, this can take anywhere from 5 minutes to 30 minutes. Sometimes, it takes me even longer.
• Swapping in the new task. The swapping in depends on the complexity of the work and how long it’s been since you last touched the task. The more complex and the longer the time since you last touched the task, and the more people you have to talk to, the longer it takes.

I don’t know how to give ballparks for each of these. Certainly, for some tasks, it’s fairly trivial. If I’m organizing a normal weekday dinner, my swapping in/out is very fast, because there’s little knowledge associated with each task. But now when I write chapters of a book (or back when I was writing code,) the costs can be very high, because the knowledge in my head is not yet written down. For me, the stopping the work is defect-inducing. Unplanned interruptions help me make defects. So does the swapping back in, if it’s been a long time since I last worked on this task.

Did I miss anything?

A few weeks ago, I republished an article originally published in Better Software: Convincing Management That Context Switching Is a Bad Idea on the AYE site. I’d received no feedback about the article when it was published, so I wanted to generate some discussion about my ideas.

I did generate a little discussion. Don Gray first said “Context switching is fun!“. Later on, he said the differences were:

* One switches between similar tasks, the other doesn’t.
* I’m not under deadline pressure. (Should anyone be?)
* I get to choose when to switch contexts.

George Dinwiddie discusses the issues of flow in his Context Switching post and the consequences of being interrupted. Don Gray in a later post discusses why managers feel it’s ok to ask people to do multiple things.

I still think managers assign people to context switch for these reasons:

• The manager can’t decide what’s most important.
• The manager doesn’t understand/remember that technical work is different from management work, and that the more things a technical person works on, the less that person will get done and the worse the work is.
• The manager can’t remember all the work required and doesn’t realize he or she is asking one person to work on more than one thing.
• The manager still believes that people who multi-task are more efficient or get more work done than people who don’t.

Do you have better arguments than mine? What do you think?

A reader sent me email with this question: “We have a group of four people (3 developers and a tester). We work on 4 products, releasing one about once a month (each product is released once a quarter). The developers are devoted to one product when they’re developing, but have to fix problems immediately if someone finds a problem and reports it. How do I manage the multi-tasking?”

I’ve asked the reader a few questions. I still don’t know everything about the environment, but here’s what I suspect is happening, based on my questions:

Each product has technical debt, from insufficient testing (all the way through development) from previous releases.

There are a ton of ways to deal with insufficient testing. I would first start with peer review of fixes. If someone is interrupted from new development to deal with an already-existing, but newly discovered problem, have the developer ask a peer to review the fix. Part of the peer review would be to create automated tests for this fix. (Yes, I realize this now interrupts two people. I find that if there’s a Big Problem in one product, there are frequently other related Big Problems in the related products.)

After peer review and automated tests, I would (gently) ask the tester how the tester missed this problem. This is for root cause analysis, not to assign blame. The tester doesn’t read or write code, so the tester can only perform black box testing. Black box testing, by its very nature, is unlikely to find most of the gotcha’s in a product. After a short root cause analysis, I would add some form of automated test development activity to each project.

I can see some heads shaking from here. “If we take time to add tests, we can’t add as many features.” You’re absolutely right. But here’s the problem. You’re not adding the features now. You’re adding broken almost-features. That’s not helping the customers. The customers deserve to have features that work.

So, to deal with multi-tasking from defects in a small group, I recommend changing the way you work. If you keep the same lifecycle you have now, add the practice of peer review of all code and all fixes. Add in automated testing for all defects that have to be fixed from the field. Change the testing, because it’s not exposing the Big Problems before you release. If you choose to change lifecycles, I suggest an agile lifecycle with test-driven development.

Whatever you do, don’t assume that wishing away the problem will make it go away. Some practices need to change. Maybe not the ones I’ve suggested, but some of them do. The change will take time, depending on how much technical debt you have. Otherwise, you’re left with the quality pledge and no muscle behind it.

Clarke Ching’s Multitasking MAKES YOU STUPID is another great article. But when I teach PMs or coach managers, they say, “I need to multitask to get things done.” Or, they say, “I’m ok with multitasking.”

Even smart people think they can do a couple of things at one time. Maybe they can. But the more things you try to focus on, the less overall you do. Think about that word focus for a few seconds. When you focus on something, you concentrate. If you’re attempting to think or work on several different things, you’re not focusing on anything.

I once worked for a company who decided its mission was to “focus on five.” Even then, I knew five was too big a number. But when I suggested we focus on two, my suggestion was dismissed. Company no longer exists.

Deciding what to do –and what not to do — is not simple. I just created a simulation (Esther reviewed it for me) to hammer home the point that we all have a limited attention span, and when we pay the necessary attention to one thing, we’re not paying attention to something else. In the simulation, people have to make choices about what to do and what not to do. I’m using the simulation for the first time next week. We’ll see how it goes. Maybe I’ll be able to report back.

If you’re faced with multitasking demands, stop for a few seconds. What’s the most strategically important work? Do that first. If you have three or seventeen things you think are all at the same importance and you can’t rank them, ask your boss. You’ll find that a bunch of the things you think you’re supposed to do may not even need to be done. Just don’t think you can do them all at the same time, because you can’t.

At a recent presentation, (Managing the Management Balancing Act) I discussed the problems of multi-tasking. I received this feedback:

Johanna, I have to say that I think you are off the path in terms of “multiple projects.”

1) Organizations just don’t work this way – it isn’t cost-effective.

2) Today’s emerging workforce (20-30) were raised in an environment that was not FIFO. It’s multi-in, multi-out.

If you can prioritize projects and sub-divide the work, these people are fantastic “multi-project” testers. Looking around the room this morning, I see many young faces, raised with multi-tasking as the norm. I think you need to update your thinking.

Oh dear. When I read this feedback, I was sure this was an unseasoned manager, assuming he/she was encountering these problems for the first time. Unfortunately, multi-tasking has been around since people had more than one thing to do :-)

In a recent Software Development article I discussed the true costs of multitasking (as have Hal and Esther and I in our blogs). This manager doesn’t seem to understand the costs; he/she sees the apparent completion of work.

As I reflected more on this, I realized that I believe each manager has the job of prioritizing the work as his/her first role. If you know what you’re supposed to deliver to the organization, you can select the people and organize the work to succeed. If you never choose what to deliver to the organization, you can never fail as a manager. Of course, you can never succeed either, because there’s always too much to do. But you can certainly never fail. Until the whole organization fails.

Managers who never reassess their work prioritization are inadequate managers. Managers who take on the work the organization requests (or demands) and who don’t prioritize are inadequate managers. Organizations that don’t rank their work will fail.

If you’re a manager, first prioritize your group’s work. Then assign it. Monitor the flow and the organization’s priorities. It’s not easy, and it’s necessary.

After my presentation last night at the Detroit PMI chapter, an attendee asked me, “Is context switching really as bad as you say it is?” Yes, it is. I believe Weinberg’s estimate of losing 10-20% of possible work-time every time you attempt to take on one more project. And, if you read Hal’s entry today, “Multi-tasking resources in contention is a principle source of throwing a project out of sequence.”

So if you’re faced with too many distinct projects, what can you do? I started to answer this here. But there’s more you can do:

1. Verify you understand the relative importance of all the projects you’re working on.
2. Verify all your work should be done.
3. Estimate how long each set of tasks will take. Bunch as many tasks together as possible.
4. If you can, spend a week on each project, so you can make progress. If you can’t spend a week, allocate days to projects. If you can’t allocate days, allocate mornings or afternoons. If you can’t do that, find another job, because this company won’t be around long enough to pay you.
5. Track your task estimates. If something starts taking longer, take a few minutes to determine why. Do you need help? Are you capable of performing the work? Could you work faster if you work with someone else?
6. If many of you are stretched over too many projects, consider pairing. Pair-work with someone on two projects, working on each one at the same time. If two of you work together on one project at the same time, you can make more progress than each of you working separately.
7. If you’re a manager, learn about project portfolio management, especially about when to fund, initiate, and staff projects. If you’re a technical person, discuss project portfolio management with your manager. Your manager may not realize what he/she is asking you to do.

Whatever you do, make sure you don’t let the context-switching and multi-tasking continue. The more spread you are, the less effective you are.

• several pieces of work at different levels causes problems (defects in code, inappropriate project plans, etc.)
• too much work, when you try to work on it simultaneously, causes you to make no discernable progress on anything

But, you’re a development/test/project manager. You have too much work to do, and not enough time to do it. What do you do?

1. Make sure you know your boss’s priorities. What does he need now? Too often the boss says everything, in which case, go to #2.
2. What is strategically important for the company now? Note that this is not strategically important for your group, but strategically important for the company. Talk to your boss, yes, but then focus on what helps the company most.
3. Kill the projects you need to kill. Too often we have too many projects hanging around, or products we support because we always have. Stop that now, and only support what is strategically important.
4. Work on things that bring in revenue (or contribute to revenue) now.

Another piece of multi-tasking is to “create” more time in your day by:

• not attending meetings you don’t need to attend
• allowing interruptions from voicemail, email, pager, cellphone, and blackberries at specific times, not when the interruptions come in
• allowing people at the lowest level of the organization make decisions about their work, so you can make decisions about your work.

I’m still thinking about this, so I’ll try to post more helpful hints later this week. (Especially once I get my connectivity worked out.)