Four R’s of Software Process Improvement: Requirements, Reviews, Retrospectives, and Results

© 2000 Johanna Rothman.

Abstract

Process improvement projects can be difficult to start, keep on track, and assess results. We can use the same requirements gathering and specification techniques that we use on product-projects on our process improvement projects. This paper discusses how to define requirements for process improvement projects and how to use reviews and retrospectives to assess the results of process improvement efforts.

Introduction

Software projects are hard to do correctly; therefore many of us have started process improvement projects to make our projects work better. Process improvement projects aren't particularly easy either. I recently saw this ad on the web:

There's a bright future ahead on the right road to process improvement. We know the right road and all the steps in it to achieve process improvement success. Our experts will guide you, making sure every step is aligned with your company's goals and everyone in the organization is headed in the same direction.

If only it were so simple. We could call in the experts, hand off our process improvement projects, and live happily ever after. Unfortunately, every process improvement project is different, because each organization has its own context and problems.

I like to apply standard project questions to discover the motivation behind my process improvement project:

  • What outcomes do you expect? What are the desired process improvement results?
  • What do you need to do to know about those results? What are the process improvement requirements?
  • How will you know you have achieved those results? What techniques will you use to review, test, and measure your process improvement project?

Use Results to Generate Requirements

Process improvement projects are supposed to create new capabilities in your organization, to affect how you perform the other projects in your company. What drives your process improvement project? What capabilities are you trying to change? Those new capabilities are your results.

Management does not always clearly state their desired results. Your managers may think their directives are the desired results. Look beyond the directives to find the real business results. I use context-free questions [1] to get at the results. Context-free questions might be:

  • Who are the clients of the process improvement project?
  • What does a highly successful solution look like?
  • What is that solution worth to you?
  • Why are these results desirable?

This is an excerpt of a conversation I had with a senior manager (SM):

SM: We want to reduce cycle time.

JR: Why?

SM: Well, we want to get our products to market faster, so we can improve our return on our product development investment.

JR: What return are you currently getting, and what return do you want?

We discussed the total costs of development and support from some previous releases and came up with a specific desired business result:

Reduce cycle time in order to double the ROI from major releases.

This is very optimistic, but at least we understood the context of the changes we would have to consider.

If you receive directives for process improvement, discuss the reasons behind those directives. Work with your management to discover the business requirements, the true desired results, and the context of your process improvement. Then create specific and measurable requirements that make sense to everyone involved–management, the product development staff, and the process improvement project team.

All too often, directives for process improvement produce sneers and cynicism from the technical staff. They are cynical because they may not believe the managers are interested in process improvement. Sometimes, technical staff perceive process improvement as a fad-of-the-month. Sometimes, the technical staff are sure this is some other thing “management” has decided would be good for workers to accomplish instead of the product development work. When you define specific and measurable business results, you can avoid or minimize much of that cynicism. Table 1 gives some examples of how you might convert mandates or directives to business results.

Table 1: From Directives to Results

Directive Cynical reaction Possible business reasons for process improvement and specific results
Be at Levelx by y Pass an audit. I wonder if those guys are from the IRS? Introduce and maintain new project management capabilities. Be able to proactively manage projects, anticipate risks. These new project capabilities would increase our ability to ship product on time, and decrease customer support costs by half.
Reduce cycle time They think we're wasting time, probably surfing the web. They just want to make us work harder. Our customers want us to ship more features faster. If we can get each release out faster, we can sell more products and increase our market share by 30% in two years.
Ship more features They want us to put the kitchen sink into every release. They think with “process”, they can make us put more features into each release. Our customers want more features, but it is difficult to work on multiple releases in parallel. If we could define appropriate lifecycles and development techniques to release more features per year, we could increase revenue by 40% per year.
Reduce costs Uh oh. We know what costs they want to reduce–us! We spend too much time on rework during each project and after the project ships. If we knew different ways to work, we could avoid generating patches and fixes, and work on new products. We could reduce maintenance costs by at least 50%, and then put the money towards new product development.

Define Your Process Improvement Requirements

Once you have specific, measurable business results–the context of your process improvement project–you can determine the rest of the process improvement project's requirements.

If your organization fails to define and manage requirements, your process improvement effort will sputter and probably fail. To succeed, you must identify your requirements, verify them against your results, manage them through the project, and then test the new process against the requirements. It is not simple, but it is the right way to improve.

Identify your process improvement requirements

After I ask the context-free questions, I define the users, attributes, and functions [2] to identify the process improvement project's requirements.

Users

Who are the favored and disfavored users? Are there primary or secondary users? Users might be the sponsoring management, potential assessors, the technical product development staff, and the product development management. You might decide that customers are not direct users of the process improvement project. You may decide to prioritize the needs of the primary and the other users. Identify the disfavored users, especially when you rank user needs. If you don't know who your disfavored users are and address their needs, your process improvement effort will fail.

One organization identified its technical staff as its primary users. The potential customer-assessors are secondary users. They are creating a process that serves the technical staff first, and serves their assessors second. This organization doesn't appear to have disfavored users. They don't have coding cowboys, royalty, or busybodies who interfere [3] with the way work is done now.

Attributes

Do you want to be able to predict better how long your projects will take? Is project performance an attribute, such as being able to support a quarterly release? Is cost to market an attribute of your projects? How testable, visible, auditable, maintainable, controllable do your projects need to be?

Another organization decided that time to market, in the form of a quarterly release, was a crucial attribute for their products. When they planned their process improvement project, they knew they couldn't complete the effort in one quarter. They wanted quarterly deliverables from their process improvement project, in the same way as they wanted quarterly deliverables from their product-projects. Part of their process improvement was culture change across the organization and across every project, including their process improvement project: from long release cycles to repeatedly delivering small chunks of functionality on a quarterly basis.

Functions

What activities does your process improvement project need to perform? Many of those activities will be the same for your product-projects as your process improvement projects: meet, communicate, negotiate, design, implement, examine, review, test, and so on. Will it be a permanent ongoing project, or will it have an end? Should you bring in the functional groups across the organization, or only across product development?

One organization working on a concurrent engineering lifecycle decided that they wanted their process to make certain the right people participate at the right time on the projects. They chose to have a specific design phase in their process improvement project, so they could discuss and test how to make design happen concurrently in their product-projects.

Another organization has made a business decision that they will not have any dedicated process people–the technical managers are responsible for defining and improving their product development processes. One attribute of their process improvement activities is that they choose to take one area for improvement and work on it for a year. That attribute affects some of their functionality, project retrospectives. Each year, they hold a project retrospective where they reassess their activities and results and decide on the next area of improvement. (I'm not advocating this approach, but it appears to be working for them.)

After identifying your process improvement requirements, you can verify them.

Verify Process Improvement Requirements

If you don't verify your requirements, you will not get the business results you want. Verifying the requirements helps you define issues and find the defects in them.

Your business requirements define the context of your problem. The user/attribute/function description defines the whole customer or stakeholder problem. Review the process improvement context and problem together. If they align, you're all set. If they don't, go back and talk to management and the rest of the people you have worked with so far.

Once you have reviewed the requirements within the process improvement project team, hold a technical requirements review with your management sponsor(s) and your process improvement team. Review the requirements with the technical staff too, as a way to make the project more real to them, and to dispel their cynicism. Use your process improvement project to show the rest of the organization how early-and-often review can improve the entire product.

Manage the process improvement requirements

Process improvement projects are subject to the same kinds of pressures as product-projects are. They are frequently under time or resource pressure, especially if the staff is not dedicated to process improvement. Someone, somewhere will want to change the requirements. Manage the process improvement project's requirements as you would manage other project change requests. The questions you ask for product-projects apply here also:

What implications does this request have on the users, attributes, and functions?

What are the schedule implications?

Will the request change our ability to meet the desired business results?

Test the process improvement requirements

Every time you define or change the process improvement requirements, test them. I prefer to test process improvement requirements in a variety of ways:

  • Verify that the requirements meet all personnel levels of process improvement and function [4] as in Table 2. I apply use cases to do this as part of the user-attribute-function matrix development and testing.

Table 2: Process improvement levels and definition

Top Management Does top management enable the process you're creating? Can they change what they do, to make this new process work?
Middle Management Does this process address how to do the work? Can they manage the work this way?
Technical leads, first line managers, and supervisors Can they follow this process to make it work? Can they create a project plan with the process embedded in it?
  • Verify that the requirements meet the desired business results with technical reviews or walkthroughs with the technical staff.
  • Inspect the requirements for defects.

Choose the appropriate verification technique for your work products.

Role of Reviews in Process Improvement

All successful process improvement initiatives incorporate reviews. Process improvement is about changing the culture of your organization, to achieve certain business results. Reviews reflect the culture of your process improvement project. When you perform walkthroughs, reviews, and inspections, you show the rest of the organization that you are willing to examine your work products. Reviews in your process improvement project will change the culture, from wherever you are now, to one that's closer to an egoless programming culture5].

Use your requirements verification activities to show how walkthroughs, reviews, and inspections can help product-projects, as well as process improvement projects. Gain consensus on the requirements with technical reviews. Inspection is an appropriate technique to discover defects in your requirements, or in any of the other work products. Walkthroughs are excellent for training purposes.

One powerful approach to performing technical reviews on requirements is to ask what's missing and why. When you ask these questions, you send very specific messages to the organization:

  • You want to know what's not working now, so you can avoid that in the future.
  • You are willing to question your own work.
  • You want to know the problems–the differences between the desired state and current state of product development.
  • You are open to changing the culture of the organization.

You show that you are willing to review your process improvement activities and process improvement project, and that you're open to change (you are not trying to be the process police). As process improvement staff, if you show that not only are you open to change, but that change is expected and desired, the rest of your organization may have less resistance to change. 6] After all, changes in results are the reasons for starting a process improvement project.

Results: How Do You Know You Achieved Them?

You have now defined and verified your desired results. How do you know when you have achieved those results? Are you meeting your users' needs? Are you meeting your defined attributes? Does your process improvement look successful?

Retrospectives

Process improvement projects can benefit from mini-retrospectives as you proceed, and from major retrospectives as you achieve major milestones. What can you learn from your progress to date? Are you keeping your business results in mind, as you define your requirements? Are you testing the process with the technical staff, to verify it meets your requirements and their requirements? Are you working in ways you want to continue, or is there a better way to do your work? What are you having trouble changing and why? Is there something you have missed?

Retrospectives model the behavior you're looking for as part of the process improvement results: to look at what you've done, and improve on the past going forward. If you're not afraid to look back at your work and learn from it, the rest of the organization will be more likely to do so also.

Measurement

Aside from in-project and post-project reviews, you can measure results. If you're not sure how or what to measure consider the Goal-Question-Metric paradigm [7] to define your measures. The goals here are your business results that you derived from the context-free questions. From your results, ask Questions that help you figure out when you have met the goal. Then, Measure the answers to those questions. Here's an example:

Goal Reduce cycle time to double ROI from major releases
Questions For the last few major releases, what are current cycle times and the current ROIs? What size were the releases, and what was the staffing level? How much time did the staff work on these projects? Did the staff have to share time between projects? Is there anything else that affects ROI?
Measures Trend the last few releases of cycle time and ROI. For each release, staff effort by week over the project. (Include other measures for any of the other ROI related questions.)

Changing your process via process improvement is supposed to produce the desired results. Use measurements to track your changes and results, and keep your process improvement project on track.

Summary

Process improvement is all about results–changing the results our organizations currently have. These results will change our culture, so we have numerous stakeholders for our process improvement projects.

Use your requirements elicitation and definition skills to derive your desired results and the requirements that will drive those results. Use reviews as a testing mechanism for your requirements and the results. Then ensure that all your stakeholders' interests intersect with your process improvement results [8].

Where appropriate, use your process improvement project to model product-project behaviors. You can test out pieces of your process on your project (sometimes called “eating your own dog food”). When you test out the pieces, focus on the desired results.

Model the behavior you want your software projects to exhibit, especially the behaviors of requirements identification and management, reviews, and retrospectives. That will help you meet your desired results.

You'll find your “right road” to process improvement, with requirements, reviews, retrospectives, and results.

Acknowledgements

I thank the following people for their valuable review comments: Elisabeth Hendrickson, Brian Lawrence, Karl Wiegers, Jerry Weinberg.

References

  1. Gause, Donald C., and Gerald M. Weinberg. Exploring Requirements, Quality Before Design, Dorset House Publishing, New York, 1989.
  2. Lawrence, Brian. “Requirements Happens…”, American Programmer, April 1997, Vol. 10. No. 4.
  3. Rothman, Johanna. “Retrain Your Code Czar”, IEEE Software, March/April 1999.
  4. Weinberg, Gerald M. Quality Software Management, Vol.4: Anticipating Change, Dorset House Publishing, New York, 1997.
  5. Weinberg, Gerald M. The Psychology of Computer Programming, Silver Anniversary Edition, Dorset House Publishing, New York, 1999.
  6. Wiegers, Karl. Creating a Software Engineering Culture, Dorset House Publishing, New York, 1996.
  7. Basili, Vic, and D. M. Weiss, “A Methodology for Collecting Valid Software Engineering Data”, IEEE Transactions on Software Engineering, Vol. SE-10, no. 6 (Nov. 1984).
  8. Wiegers, Karl. Software Requirements, Microsoft Press, Redmond, WA, 1999.

Like this article? See the other articles. Or, look at my workshops, so you can see how to use advice like this where you work.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.