Software Quality Assurance: Should It Remain a Separate Organization?

© 1996 Johanna Rothman. This article was originally published in SQA Quarterly, May 1996.

Product development teams are organized for one major purpose: to produce a product people will buy. Software product development teams have a secondary, but no less important goal- the ability to produce products again and again.

To effect those goals, product development organizations typically are organized into either functional groups or project groups. Especially in software engineering, many organizations have a hard dividing line between product development and the SQA or product verification functions. This paper will discuss the roles of software engineering, and the benefits and disadvantages of separete and merged development and SQA organizations.

Background: What is Software Engineering?

For this discussion, let’s define software engineering as all of the tasks needed to produce a quality product: define requirements, design and implementation of product and tests, product verification, and release creation. The goal of software engineering then, is to make money for the company by producing products people want to buy.

Software engineering personnel must determine what quality means to the customers, define how to produce the product (the product development process), design, implement, test, and ship the product. In addition, the software engineers must be able to learn from the mistakes made on previous projects.

Many organizations define quality as just the measurement of low defects. However, that is a very limiting definition. To really define quality, one must also consider functionality, usability, reliability, performance, and supportability [Grady92]. Once quality is defined for a project, one can make the process and project tradeoffs accurately.

Software Engineering Roles

Let’s examine the roles of software developers and software quality assurance people. Software developers define requirements; design and implement the product; and minimally verify the product meets requirements. Developers frequently use functional specs, design documents, and prototypes to help define requirements. When developers design and implement the code, they write detailed designs, write the code, compile, and unit test the code. Frequently, developers also verify the product via integration testing.

Software Quality Assurance people, the product verifiers, define product test requirements; design, implement and verify the tests; verify the product meets the requirements. SQA engineers use testing specs and test scenario designs to help define the test and product requirements. When SQA engineers design, implement, and verify the tests, they write detailed scenarios, write the code and/or test procedures, compile and verify the test code. They verify the product meets the requirements via system test- by testing all functionality of the product in a systematic way.

There are significant similarities in these processes: everyone defines requirements, designs and implements product or test code, and verifies their development activities. The output focus of the product developers is on shippable product- the code that ships. The output focus of the product verifiers is on non-shippable product: the code (tests) that stays.

Aside from the process similarities, there are significant differences in the engineers’ focus in these activities. Developers tend to be area specialists- they understand and develop a narrow area of the product, and understand it fully. SQA engineers tend to have a more general perspective- they understand the whole product and gather data to understand the product state. So, even though the process is very similar, the focus of the work is quite different.

The difference in focus sometimes leads to a difference in ability in technical skills, especially in the testing areas. Unit test development requires product code knowledge, what code coverage means and how to measure it. Integration tests require understanding the configuration management system and how to assemble the product correctly. In addition, integration testing requires an architectural knowledge of the system, so that these first-line tests can be properly assembled. In addition to the variety of techniques, system test development requires product and code knowledge to be able to look for and test error-prone areas.

Even with these differences, the similarities between product developers and product verifiers is striking. Since the goals of software product development are not just to produce one product, but to repeat successes, can we organize software product development to utilize these similar processes between development and verification?

Functionally-Organized Product Development

When product development is organized by function, the developers, SQA, documentation, tools, release engineering, are all separate groups. There are more developers, or development groups, so, by default, the development groups own the product development methodology. The developers define what they do and what the other groups’ responsibilities are, and the developers are the driving force behind product development. Typically, the SQA groups owns the verification and release processes. SQA defines what gets tested and when, and what the shipment criteria are. See Figure 1 for a view of the traditional organization.

Fig1SQAseparate

Figure 1: Traditional Organization View

There are a number of benefits to a functional organization:

  1. The product author and testers are independent at least at the technical contributor level.
  2. Managers understand how to grow the organization.

There are a number of problems with a functional organization:

  1. SQA gets involved late in the project cycle.
  2. Even when an overall project leader is chosen, there is no incentive for functional managers to cooperatively work on product development process improvement.
  3. There is no cross-functional mentoring or growth.

Because of these problems, most functional organizations choose to revamp their organizations when faced with process concerns or process improvement.

I have had experience with two types of product development organizations not organized by function: Functional managers with empowered teams; and self-organizing product teams.

Functional managers, Empowered Teams

In this model, there are still separate functional areas: Development manager, SQA manager, and their staffs. Management owns the overall methodology, tools, process issues and work. The team owns the product development, documentation, and verification. The idea is to create small project groups within a larger project, and allow the technical contributors to choose the roles that suit them best. See Figure 2.

Figr2SQAseparateFigure 2. Functional Managers, Empowered Teams

This is quite similar to traditional matrix management, except that in traditional matrix management, the managers assign the roles to the teams. Empowered teams choose appopriate roles for each team member.

Team members define their roles, and how to do their work. Managers mentor people when they have issues. There are still centralized groups, if they do work that supports a number of groups, such as tools development.

The benefits to this kind of organization are:

1. The team thinks of problems as system issues, and solves them in systemic ways. Systems thinking is a powerful problem solving tool, and these teams tend to use it more often.

2. Managers have the ability to keep corporate history of projects more easily.

3. This provides people an easy way to do career development. If the team sees that the person is capable of more, the person will get more to do. If the person is not as capable, the person will get less to do.

4. The project members define how to exactly work within the corporate product development process- they can define how to use the process in a way that fits their needs.

The problems with empowered teams and functional managers are:

  1. This type of organization is unstable because it does not meet the control needs of many managers.
  2. Managers can get lazy and not keep corporate history.
  3. Technical contributors can be pigeonholed into one type of role if management has too much input into the role definition.
  4. This does not scale to small organizations (about 30 people or less).

Self-organizing Product Teams

An organization with self-organizing product teams has managers who function as project leaders. Their management functions are in the areas of career development and and in getting good projects for the team to do. See Figure 3. The key here is that the project leader does not hold all the information- the project knowledge is shared among team members.

Fig3SQAseparate

Figure 3: Self-Organizing Product Teams

Team members must trust each other, to share knowledge and to have sufficient abilities to actually complete the work. Frequently, these team members choose areas in which to concentrate by component. The team members may trade off roles (see Table 1), or may use a buddy system to do the work (see Table 2).

Table 1: Examples of team assignments – role tradeoffs

Module Product Code Documentation Test Code Tool Code
A Jo Jim Jack Jill
B Jill Jo Jim Jack
C Jack Jill Jo Jim
D Jill Jo Jim Jack

In this example, everyone trades off on all tasks. This is not common- generally, different people have different talents and roles, and so they will want to work on different things. In that case, the buddy system may be more effective.

Table 2: Examples of team assignments – buddy system

Module Product Code Documentation Test Code Tool Code
A Sally Sue Sam Stone
B Sue Sally Stone Sam
C Sally Sam Sue Stone
D Sam Sally Stone Sue

Using a buddy system, two people work together on a set of tasks- here either product code/documentation, or test code/tool code. Depending on the module, some people may even change roles, but still work with a buddy on some modules.

The benefits to this kind of organization are:

  1. In addition to solving problems in a systemic way, the team has the ability to learn, both about its processes and the product. In addition to team learning, the individuals can choose which specific areas to learn about.
  2. The team, in addition to the individuals, has more potential for flow, the ability to enter a highly productive product development state.
  3. The team can develop its tools and methodology quickly.
  4. The team can define how to use the process in a way that fits their needs.
  5. These teams are completely self-empowered.

The problems with empowered teams and functional managers are:

  1. This type of organization is also unstable because it does not meet the control needs of many managers.
  2. Even if the managers keep a corporate history, the team must have a way of passing on its knowledge. There is no clear answer for how a successful team can pass on its success to others. And, if the team breaks up, how can the team members pass on their knowledge?
  3. If the team is not competent, or does know the boundaries of its knowledge, the team can fail.
  4. Technical contributors can be pigeonholed into one type of role if management has too much input into the role definition.
  5. This may not scale to larger organizations (about 70 people or more).

Conclusions

Product development organizations focused on developing quality products that their customers will buy need to consider the organizational structure that works best for them. It is not an organizational requirement to separate the tasks and people into functional groups. The product may require a project-focused group, focused on the product, not the organizational hierarchy.

When product development groups consider their organizational needs, they need to consider their staff, quality requirements, the process requirements, and the kinds of projects they have. In many cases, integrating the product developers and product verifiers onto a project team will have multiple positive effects in terms of project schedule, product quality, and increase in team knowledge. As the team increases its knowledge, management can trust the team more to meet schedules and deliver the promised product.

Especially as company products evolve from startup to mature product, the management team must have a flexible approach to project organization. Even when management has been blindingly successful for a particular level of achievement, what got you here may not get you any farther.

Like this article? See the other articles. Or, look at my workshops, so you can see how to use advice like this where you work.

Leave a Reply