The Role of the Test Manager

Test managers really serve two very different customers, their testers and corporate management. For the testers, the test manager helps develop product test strategies, and provides test expertise to the testing group. For management, the test manager gathers product information so that corporate management can decide when the product is ready to ship. For both the testers and management, the test manager helps define and verify product ship criteria.

For the Testers and Managers: Define and Verify Product Ship Criteria

As the test manager, you have an opportunity to negotiate the criteria with marketing and the development groups. You want to verify criteria with customers or representatives of the customer. The development group's job is to figure out how to achieve what the company wants to achieve. Using the customer requirements to figure out what to do provides the developers a complete view of the product and how it works. Once the product is defined in terms of what and how, the testing effort can verify how well the product meets the real requirements.

It's important for testers to prioritize their testing so that the product ship criteria can be met. Since very few projects have enough time to do everything, getting the testers enough information for what and when to test is a very important role for the test manager.

Corporate managers need to understand the product ship criteria enough to be able to judge whether the product is ready to ship at a given date. I don't believe the test group should be either the holder of product approval or rejection-that role belongs to corporate management. Having pre-negotiated, agreed-upon ship criteria helps corporate managers judge for themselves whether or not a product is ready to ship. Pre-negotiated criteria helps the project team decide what product quality is when one no one is stressed from the project work.

For the Testers: Develop Test Strategies

These ship criteria should have a significant impact on the test strategy for the product. For example, if the product is an operating system that has to run on multiple hardware platforms, ship criteria that emphasize the number of platforms the product has to run on, and de-emphasize the specific features of the operating system will push your strategy to one of hardware combination testing, not depth-first testing across many features on one particular platform. The test manager does not necessarily develop the entire strategy for testing, but should have input to the strategy, and participate in reviewing the strategy with the test group.

For the Testers: Provide Test Expertise to Test Group

The test manager should provide some test expertise to the test group. This expertise comes in a variety of forms:

Ability to discuss, mentor, and/or train current employees on general testing techniques.
Ability to look forward to the products in development and planned for development, to foresee what new expertise and training is needed by the group.
Ability to hire people based on current and future testing needs.

Note that this does not mean the test manager has to provide specific low-level tests for the product. A test lead's job is developing tests, not the test manager. As a test manager, you provide leverage to all the testers and to corporate management. You need to define for yourself the difference between mentoring and doing other people's work.

For the Managers: Gather Product Information

The successful test manager also gathers product information, in the form of defect counts, test pass rates, and other meaningful data. The test manager defines the data, and then gathers it for presentation to corporate management. For example, I gather what I consider to be the "basic" metrics:

Defect find and close rates by week, normalized against level of effort (are we finding defects, and can developers keep up with the number found and the ones necessary to fix?)
Number of tests planned, run, passed by week (do we know what we have to test, and are we able to do so?)
Defects found per activity vs. total defects found (which activities find the most defects?)
Schedule estimates vs. actuals (will we make the dates, and how well do we estimate?)
People on the project, planned vs. actual by week or month (do we have the people we need when we need them?)
Major and minor requirements changes (do we know what we have to do, and does it change?)

Conclusions

The test manager bridges the gap between what corporate management wants and the current product state, not by being a policeman, but by helping

The different groups articulate what the product has to do for it to ship
To plan the strategy to verify the product can do what it has to do to ship
To assess how well the product does what it is supposed to do.

Johanna Rothman observes and consults on managing high technology product development, looking for the leverage points that will make her clients more successful. You can reach her at jr@jrothman.com or by visiting www.jrothman.com.

What can we do for you? Go to: Writings and Presentations, Chronological Listing