Software product development is a difficult task. Not only is it mentally challenging, just to write and test software, but there are a number of interdependent problems when product teams attempt to create a product. Product teams have difficulties in these major areas: meeting the schedule, implementing the desired functionality, and removing enough defects to satisfy the customer.
To help reduce the product development problems, some organizations have taken steps to define their processes,. Many of these process definitions and steps focus their efforts on defect removal. However, the particular actions taken to positively affect defect removal may have unintended consequences for the other problem areas. In addition, actions taken in the scheduling or functionality areas may have inappropriate effects on defects. The goal of any software project is to choose the most effective activities to successfully produce the software product.
It is possible to characterize product development efforts to determine which activities will have the most overall positive effects. This paper will illuminate the issues in choosing those product development activities. We will discuss the problems in the typical development cycle, and how to characterize organizations and their products for process improvement. Then, we will examine the consequences of different actions, and how some actions performed for one area of improvement can positively affect other areas.
Product Development Focus
During a product's life, there are different focal points for the customers' perceptions of quality. Grady [Grady92] claims there are only three quality goals: minimize the schedule, maximize the features, and minimize product defects. Moore [Moore91] shows the different phases a of a product's lifecycle. Table 1 shows the interaction between product lifecycle and customer needs for quality.
Table 1: Interaction between Quality and Product Lifecycle
During a product introduction, the product must solve enough of the customers' problems, but does not have to be fully featured. The product does have to get into its customers hands quickly, or a competitor will take away the market. Once there is some product acceptance, features become more much more important, and time to market is still critical. Once the product has taken over its marketplace, customers are much more interested in having a well-running, predictable product, so low defects is critical. And, if a product does not maintain a low level of defects, its “cash-cow” position in the market will be limited.
Given that customers see the product differently as the product matures, the product development processes should evolve as the product evolves. In the table above, there are no cases where all facets of quality have to be satisfied in any one release. There is only one case where there are two possible high priorities, and even then, a particular product can generally choose one of the high priorities as the top priority.
Product development actions have different effects on the project. Figure 1 is the overall picture of the software product development process actions and their effects.
Figure 1: Effects of Priority on Project Tasks
As defects are removed, and features implemented, and as a project continues, the cost and time estimates are updated, as well as the perceived project status. According to Abdel-Madnick, [sw proj dynamics], software projects are always late. As the project delays become more obvious, corporate management often chooses to change project priority. However, changing priorities delays the project. Up until now, the project had been organized and optimized along one of three lines- now that optimization is being reduced and a new priority is introduced.
Figure 2 shows the actions and effects of using defect removal as a top priority. Designs and code are reviewed, unit tests are developed, and frequent build and test runs are implemented. Defects can be counted and tracked easily, and the defects are removed. If reasonable metrics are used, the time and cost left to complete the project can be well estimated once the project is in the late development or early integration phase.
Figure 3 shows the actions taken and effects when the focus is to maximize product features. Customers review the product designs, developers generate prototypes for customer review, and a requirements verification activity occurs, to work the requirements issues with the customers and producers. As features are implemented, the project cost and estimates are updated. However, it is not clear how to continue estimating how much longer a given feature will take to develop. The design phase may take longer, and if the testing phase does not start early, it can delay the whole project.
Figure 4 shows the actions and effects of minimizing the schedule. Prototype code and automated system tests are developed in an iterative fashion. To know where the code (project) is at any given time, a nightly build and test mechanism may be used. To meet the schedule, prototype code is sometimes shipped as product code, but if the test mechanisms are sound, that may be an acceptable risk.
As you can see, each quality strategy has different actions which cause different effects. If a project is planned with low defects as the premier goal, and that goal changes to time-to-market, the project needs to be replanned with different activities.
During an initial release of a new-paradigm product, just before the final beta test period, product goals were changed three times by management. The original goals were first: low defects (as measured by development staff); then time to market (as measured by management); then product performance (as stated by management).
A number of interesting events occurred:
- Each new goal was in effect for no more than 5 weeks.
- Even though the professed goals changed, the effective actions taken by development staff did not change. That is, the lifecycle methodology and software engineering practices, some of the actions as described in Figures 2, 3, and 4 did not change to accommodate any of the new goals. The development staff's actions were still based on fixing enough bugs to make a market ship date. There were a number of discussions and arguments, but the development staff were not capable of replanning the project to take different actions. In fact, the staff's notions of product quality were so ingrained, they were convinced there were no different actions to take.
- The number of bugs fixed continued to increase.
- The ship date slipped at least two weeks every time the goal changed.
- Using the original product shipment criteria, the product shipped 5 weeks late. This delay was caused by the changing priorities.
At the time of the original goal changes, the project was planned to run system tests, find bugs, report bugs, and fix bugs. This cycle took a minimum of 3 days to run. The original product development process had these characteristics:
- A slightly concurrent lifecycle: while the system tests were run, the developers were fixing bugs.
- A nightly build and test run.
- Automated system tests.
- A development focus on defects, and a management focus on schedule.
All of these activities are appropriate for either minimizing the schedule or removing defects. There are other necessary activities to be most effective at either minimizing the schedule or defects, but these activities are required.
There was significant management pressure to find bugs faster and fix more bugs. The developers chose to institute code walkthroughs to find and fix more bugs per test cycle. This triggered a delay in the start of the system test runs. The result of this delay was finding and fixing fewer bugs (the left-most cycle in Figure 5).
Since there were fewer bugs being fixed, and management realized the customer wanted more features, they changed the release goal to add in “just a few small features”. See the middle cycle in Figure 5. Those of us in the software business recognize this as “feature creep”. Since the development staff was still dedicated to finding and eradicating bugs, they chose to design and hold design reviews; to code, write some minimal unit tests and hold code walkthroughs. These actions are all good software engineering practices, and will result in code with fewer bugs. However, these activities also increased the delay in getting the features into the testable builds, and the delay in getting the product to market. These review activities are required for minimizing defects, but show the negative interaction with the schedule. As the defect minimizing activities increase, the schedule minimizing activities decrease.
Since the product was not yet available, management became even more concerned with product performance. The longer the delay to market, the more important performance becomes. You can see this in Figure 1, where as the product market becomes more mature, the customer perceptions of quality change. If the market has already moved to initial acceptance, then features, including performance, vie with minimizing schedule as the top priority as the product development goal.
Now, management wanted to minimize the schedule and improve product performance. The developers applied what they considered to be appropriate software engineering practices to the problem of measuring and improving performance: analysis of current performance, design and design review of product changes, code walkthroughs, and some unit test development. There was a double negative effect on the schedule from the performance changes to the product- the changes take time to implement, and they took people away from the bug fixing work. So, the delays in the schedule introduced by this goal change were magnified.
Management realized that with all the goal changes, they were causing delays in the project. The product development staff would have planned the project differently if the original goals had been well defined, but they were not capable of replanning with different priorities and maintaining the original schedule. Management reverted to the original quality goals and shipment criteria. The product shipped, only 5 weeks late.
A causal diagram to show all of the interdependencies is too complex to show here, so Table 2 is a list of interdependency causes and effects. There is clearly more work to be done to develop a dynamic model to show the changes in a software project.
These causes and effects explain why software product development organizations are most focused on defect removal- almost all activities have a direct positive effect on defect removal. Even the two activities that do not have a direct positive effect, have a subsidiary effect on removing defects. Requirements verification causes the development team to know more precisely what they are developing. This is a defect prevention activity, but does not directly cause defect removal. In the same way, iterative design and development causes the product development team to know what they are developing and what the issues are, but does not directly contribute to defect removal. In fact, if prototype code is shipped, defects increase.
Software project planners need to address the schedule, features, and defects when planning the projects. It is critical to consider the product's place in the marketplace and the general marketplace when defining the quality goals. It is necessary to prioritize the goals, so that the appropriate software engineering practices and lifecycle can be chosen and used effectively. When conditions change, articulate the necessary changes to the lifecycle and practices, so that the goal changes are reflected in the practices. And, make sure that management especially recognize the effect of changing goals late in the project- schedule slips.
Critical thinking, especially systems thinking, is required for software project and corporate management. If project and corporate management consider their options, including the causes and effects of those options, they can improve their ability to implement features, reduce product defects, and meet project schedules.
Abdel-Madnick, Tarek and Stuart Madnick. “Software Project Dynamics, An Integrated Approach”. Prentice Hall, Englewood Cliffs, NJ. 1991.
Grady, Robert. “Practical Software Metrics for Project Management and Process Improvement”. Prentice Hall, Englewood Cliffs, NJ. 1992.
© 1996 Johanna Rothman.
Like this article? See the other articles. Or, look at my workshops, so you can see how to use advice like this where you work.