The Influential Test Manager
© 2000 Johanna Rothman. This article was originally published in Software Testing and Quality Engineering, March/April 2000.
Many of us have worked in test groups in which we felt as if we didn’t have enough time, hardware, or staff to do the work. In those situations it’s hard to escape the feeling that while somebody might be in control, we are certainly not. As a test manager you don’t have to work this way. There are other, more effective ways to develop and use your influence within your organization to help your test group–and project–succeed.
Influence is not about control, nor is it about taking away other peoples’ choices. Influence is about coming to a mutually valuable decision about a problem–redirecting your organization’s power and focus. As a test manager, you may want to influence others in the organization when:
- Your management wants to release a product you know is full of defects, and you’re sure the customers will be disappointed with the release.
- Other managers ask you to give up equipment your test group needs to do their jobs, or you can’t get the equipment you need.
- Schedule slips shorten your test schedule, and your only alternative seems to be to test less.
- Your budget isn’t large enough to hire the people you need.
These are examples of tactical problems in obtaining specific objectives. You can also exercise influence more broadly–strategically–to change how the organization perceives itself, or to change how it does business.
As test managers we can be valuable agents of change. We have a different perspective on these problems than other managers in our organizations, because of our role in planning and measuring testing to obtain and share information about the product under test. If testers, as James Bach says, “hold the flashlight,” then test managers figure out at least some of the places to shine that flashlight, and how long to spend flashing–a specific form of risk management. Because we illuminate risk from a different perspective than anyone else on the project, we often have a unique approach to problem solving.
We can influence other people to see our perspective and agree to it, or to come up with an even better solution to our problem.
What is influence?
A dictionary definition of influence is:
“The act or power of producing an effect without apparent exertion of force or direct exercise of command”
“The power or capacity of causing an effect in indirect or intangible ways: sway.”
To sway someone, you get that person to change what they’re doing. Sometimes you give them something they want–their WIIFM (What’s In It For Me). Sometimes, you get them to reconsider their WIIFM, to enlarge their wants to include more of the organization. To be truly influential, first you must become fully aware of your problem context–the setting or environment where the problem exists, including all those who might be affected by it. Once you’ve surmised the problem context, you can decide how to influence the other person. (You might even change your perspective, based on your understanding of the problem context.)
Consider how you can provide value, and finally, what your “influencee” wants. Then, you can choose how to provide that value and what to ask for in return. This value exchange is the essence of influence.
Note that I’m not talking about a strictly monetary exchange. Sometimes you loan your credibility to some project your influencee wants to succeed. Sometimes you give your time to some other group, or perform some other service to the organization. Sometimes you get a budget increase to give more value to the organization. View your problem within the context of your organization, so you can see what you want and what you’re willing to give.
Share the “problem context”
Many test managers don’t have the staff they need to do an effective job, nor do they have the budget to hire the staff. One way to define that problem is:
I don’t have enough testing staff.
That problem statement is a good start, but is limited in scope and shows no way for you to provide value to more of the organization. An alternative problem statement is:
I don’t have enough people to execute my mission.
That’s vague as it stands, because it depends on what your mission is. Your mission could be: “Find and report defects, especially Big Bad Bugs.” Or your mission could be: “Test this software beast until they pry it from my unwilling fingers, finding and reporting on all defects.” There are many possible test manager missions–maybe as many as there are test managers! I choose to define the test manager’s mission as:
To assess the state of the product under development at any time and report on that state.
With this mission statement, I can describe the problem context as:
Without more people to do the testing, I am unable to sufficiently assess and report on the state of the product as well as I would like. I can’t give you, Project Manager or Senior Management, the picture I think you need to see. That means you, the management team, will make decisions with sketchy information. You can still make the decisions, but you are assuming risk. I don’t know how to give you better information without more people. If you want to take the risk of operating with very incomplete information, that’s okay. But I recommend we get more staff so you don’t have to make blind decisions.
This problem context statement brings the problem’s significance to the rest of the organization. It shows that the problem is bigger than the test manager whining about not enough people. It makes inadequate test staffing an organizational issue, not a testing issue. Now that you’ve shared the problem context, consider what youhave of value to the rest of the organization.
What do you have of value?
You provide value both as a person and as a test manager. Your perceived credibility, fairness, track record, and capabilities as a manager have value not just to the test group, but to your peers, your managers, and the rest of the organization.
Your own level of test-manager-value depends on how you define your mission as a test manager. Your mission defines the work you choose to accomplish: with the test organization as a whole, with the project manager, through your mentoring of staff, and in the way you structure your team’s work. When you perform that mission, you have at least these valuable items:
- Information about the product under development or test
- People who test
- Machines that execute the tests
- Some number and type of test procedures (e.g. automated and manual, complete tests or smoke tests)
Different people use their value differently, even when going after the same goal (getting funding for more testers, to provide more information about defects to the organization). Here are stories of three test managers and their approaches to getting more positions funded in their organizations.
was a test manager who also ran the customer support group and the technical writing group, in addition to the test group. She had two testers, four technical support reps and two writers. (Her organization had six developers on staff.) Andrea needed at least two more testers immediately, and wanted an additional two automation and tools developers over the next year. Andrea’s approach was to tell her boss, the Vice President of Engineering, exactly what was necessary: “I need two more testers right now, and two more over the next year.”
The Vice President’s response was bottom-line-based: “How expensive are those testers? Every time you ask me for more people, it costs me money–as much money as a developer. We’re a young company, and we have to make our balance sheet look good. Otherwise our investors will be upset, and we’ll be in trouble. Live with what you’ve got.”
Andrea and her boss have had this same conversation at least half a dozen times. Every few months the development team adds more people to its staff, but Andrea can only add tech support people, because they cost less. As the number of developers has increased, so has the software’s complexity and the size. Andreadoesn’t have enough staff to test the product sufficiently, so the company is building up its product technical debt.
[Technical debt, as defined by my colleague Dave Smith, is the debt a company "owes" to a product they persisted in shipping in an incomplete or unstable condition over several releases. This is a situation referred to by Hunt and Thomas's book The Pragmatic Programmer as "software rot"; but the software isn't necessarily rotten--it's just incomplete.]
At some point, as the technical debt increases, the load on the customer support staff will be so overwhelming that the company will have to use developers to staff the phones and solve the support problems. Andrea can see this coming, and knows there is a window of opportunity before the “technical debt overload” overwhelms the company’s ability to do development–and actually prevents new product development. She knows she needs more testers to prevent this from happening. But she doesn’t know how to explain the significance of what she’s seeing to her management. Even though Andrea can see what’s happening, she hasn’t defined her organizational value in a way that would allow her to address the complete problem.
has approached the technical debt overload problem differently. Bill also has responsibility for both a test group and the technical support group. In his organization, the writers are part of the development team. Bill already has five testers, including one full-time test tool and automation developer. (There are ten product developers.) Bill wants to hire two more testers–because two releases ago, the test group fell behind the developers and significant parts of the product were not fully tested. (Technical debt is starting to affect Bill’s organization. The testers don’t feel as if they are fully testing the product, and the tech support folks are hearing about the same defects repeatedly.)
Bill used the assessment and reporting mission statement to explain the problem context, the problem significance, and his value to the situation:
If I can get two more people to do the testing, I will be able to assess and report on the state of Modules A and B. Since the product is evolving to be based more on Modules A and B, I will be able to provide management the information to understand the development progress, and make ship decisions. If I get two testers now, I can do enough testing of Modules A and B to avoid incurring more technical debt, and having to keep people testing the inevitable fixes after we ship the product. I can use this release’s testing as an early warning signal to anticipate the load on the technical support group.
Bill had a two-pronged approach. First, Bill went to the Sales Vice President and asked if she needed technical pre-sales support. (He knew she did, because the sales reps constantly called the support reps for help before they went on sales calls.) When she said yes, he made an offer: “If you support my request for more testers, I can give you phone-based pre-sales support after the product ships.” (More testers helped find defects before the product shipped. The testers could focus on the next release, and the support staff wasn’t kept busy finding and reporting already-known defects.) The Sales Vice President was thrilled, and agreed to support his cause in the Senior Management meeting.
Then Bill went to his boss, the Engineering Vice President, and made another offer: that if he got two more people in testing he could provide benefits across the organization.
- Engineering would know about the defects earlier. That would help them choose what to fix and when.
- Knowing about the defects would help the testers react quickly to areas of the product, choosing where to do more of which kinds of testing.
- Bill would be able to anticipate the support load, and possibly help the company avoid having to hire more people for support.
- The support staff could spend more time duplicating new customer-reported defects–which would help the developers–instead of writing up more already-known defects.
He said, “If you sign two more testing requisitions, and help me hire more testers before the freeze milestone, I expect to be able to provide pre-sales phone support to our sales staff. That should increase our sales from this product.”
Bill’s boss said he’d bring it up in the budget meeting, and was pleasantly surprised that the Sales Vice President supported Bill’s request. Bill got his requisitions, and was able to hire the people he needed.
Andrea’s value to the organization is more limited than Bill’s. Andrea is focused on the testing part of the problem, which is not sufficient for showing the value she provides the organization. Bill realized he could use his staff in a number of ways, all of which would provide overall value to the organization. Another test manager, Carol, took a different approach, focusing on a narrower scope of influence than Bill: the product development organization.
was recently hired to run a software test group of four people. Those four people attempt to test the product that sixteen developers are working on. Carol realized during her interview process that this organization had enormous technical product debt. Carol decided to investigate the implications of the technical debt. She talked to the other managers and listed two major consequences of the inadequate testing: developers were spending almost half their time on product support, and the last two product releases missed their ship dates by 100%. (Since they were working on 6-month releases, 6-month slips were disastrous.)
Carol went to her boss (who was also the development manager) with a proposal: “I have some ideas about how to meet our desired six-month release dates, and how to reduce the time the developers spend supporting the product.” That got her boss’s attention. She laid out a plan that included hiring a project manager, peer review of bug fixes, and hiring four more people with a variety of skills for the test group.
Carol chose to exercise her value inside the product development organization’more than just the test group, but not across organizational lines in the company. Carol is showing she has more than just “manage-the-testing” value to the organization.
Provide Value and Recognize What Others Want
To use influence, you provide value and help others see the significance of the problem. Then it’s time to look outward to see what other people want. What is valuable for you to provide?
You provide value to your organization in many ways. Sometimes, it’s your general managerial assets that others value: your credibility, fairness, track record, and management capabilities. In addition to your management capabilities, you can provide additional value by:
- Assessing risk, starting at the beginning of the project and continuing through to the end
- Identifying and solving problems that prevent testers and developers, all of Product Development from making progress
- Managing the test and measurement work (for both the product and the project) to assess the product state, report on it, and see if it needs improvement
- Helping make ship decisions
You are much more than just someone who manages or leads the test effort on a product. If you can see the significance of an issue, and can share that with the rest of the organization, you are incredibly valuable to your organization. One way to share significance of issues is to assess and report on risk.
Product testing and measurement is a primary vehicle for the test manager to help assess risk during the project. The risk isn’t just limited to the testing organization, where a risk might be “We can’t get the testing we’d planned to do done on time.” The risk exists across the product development organization, and sometimes across the company. The three test managers above recognized that the risk to early shipment of inadequately tested product was higher-than-desired support costs, and less capability in development for the next release.
To provide value in risk assessment, it’s a good idea to look beyond the boundaries of the test group to see who else is affected by creation of technical debt, or by slipping schedules. Technical debt, for example, affects developers’ ability to adequately design the next round of changes (and creates an unstable base on which to implement those changes), and also has an impact on Support and Sales.
Identifying and solving problems
Here’s a problem many test managers run into: the developers miss their deadlines, and your time in test is reduced. Before you even start testing you know there is a substantial risk that the product that will ship is not the product your organization had planned to ship. If you can identify that problem early to the project manager, you may be able to influence the solution.
One solution could be to ship the product as an early release, such as a Beta version, to give you more test time. Another solution could be to change how the developers fix the defects, or to target certain areas of the system for defect fixing and defer fixes in other areas. Another possibility would be to reduce the number or scope of the features to be shipped in this release. The solutions you develop will depend on why your company is shipping the product.
Managing the test and measurement work to assess product state
The testing results help all the decision makers evaluate risk. As the test manager, you look for information about the product in order to answer a number of questions:
- Did we build the product we intended to build? When you test the product against the requirements and the design, including functional, performance, and reliability testing, what do you find?
- How much of the product have we tested? Does it make sense to ask this question?
- How stable is the product at this point in the testing? I expect the product to be very unstable during the beginning and middle of the project. I expect it to become more stable as we approach the final milestones, especially the ship date. What happens if the product does not become more stable?
- How complete is our information? How good is the information we have now? How wise is it to make risk assessments based on this information?
You may have other product- or project-specific questions to add to your list. Choose questions that help you answer whether you’ve accomplished your goals and priorities. I tend to choose measures that help me know when we can ship the product.
Helping make ship decisions
“Can we ship this product yet?” If you can help your organization make that ship decision, you’ve satisfied many people’s WIIFMs.
It is difficult for many of us to make product ship decisions. As test managers, some of us want to keep the product in test until they physically pull it away from us. As project managers or development managers, we may have bonuses riding on release dates (a foolish, but very common idea). But if we are responsible for the tech support work, we have even more motivation to find and fix more of the defects before the product ships.
No matter what our perspective is, we may think we know what the right ship decision is. It’s not possible, however, for the test manager alone to make a ship decision without having negotiated and objectively defined release criteria with the rest of the project team. At the very least that criteria has to define what the organization is looking for from this product for this release, and may be an excellent way to identify leverage points for what other people want.. (For a brief discussion of how to create release criteria see Rothman and Lawrence.)
What do you want in return?
For the purposes of this discussion I’m going to assume you don’t work in a “corrupt” organization, in which managers exploit their positions just because they can. Using influence on those people requires that you look at their personal positions in the organization and satisfy some of their selfishness. I find that placating those people gives me heartburn, so I choose not to work with them.
Instead, I’m assuming you work with reasonable, well-intentioned project and senior managers who Deming was talking about in Out of the Crisis: “No matter how it looks, everyone is doing their best.” They might not know how to use test managers, or they might be making some classic testing mistakes themselves (such as assuming you, the test manager, are somehow responsible for quality).
Your managers and colleagues aren’t stupid. They know they need test managers, and they know they need the product tested. But they may not understand how to translate that abstract “knowing” into how they can use you more effectively. When you influence people, you help them realize what you can provide them. Now you can use your value to decide what you want in return from the organization.
When you help define the consequences of releasing a product with technical debt…or when you define and track release criteria…or when you have conversations about what the organization requires from the release…you’ve provided substantial value. You may get some well-deserved resources or support by using that value to illustrate to your managers:
- Why shipping a defect-laden product increases other groups’ pain
- Why funding the test group’s equipment requests will help the other managers know sooner when the product is ready to release
- Why shortening your test schedule will prevent you from collecting enough data to know whether to ship
- Why you should hire enough people to do the right amount of testing so everyone will know when to ship
You may want something else in return. That’s fine. Whatever your goals are, just be clear on what you want to get from this two-way relationship.
When It’s Not You
At times, when we attempt to provide this value to the organization, we feel like it’s us against the rest of the world. That’s not a good place to start when you want to influence people. Let’s examine why you might feel that way:
- If you’ve somehow gotten yourself into the position as the person responsiblefor quality in the product. This, as Marick points out in Classic Testing Mistakes, is a classic testing mistake. If the people in your organization want to hold you personally responsible for the quality of the product, say, “Here’s how I canhelp you make that quality happen in the product.” It’s not your job to uphold some arbitrary quality standard just because they say it’s so. Work with the other people to define the value the test group provides to the rest of the company. If other people in the company have a quality standard they want upheld, the developers should uphold it. This might lead other people to say “That test manager just isn’t a team player.” But you can be the best sort of team player by asking the important questions: why the quality standard is important to the organization, what results they want to see, and what a successful release looks like.
- If you’re the only one assessing product shipment risk and other situations where you, and you alone, can see the implications of shipping this product on the business. If you’re the only one with the data, and you’re the only one to make all the decisions, Murphy’s Law says you’re bound to make some bad decisions. Murphy’s Law holds true at other times too, but here you have a single point of failure, which is much higher risk. You also may not be aware of the business decisions involved with this product release.
- Maybe you’re just having a bad-manager day. That’s okay. Wait to work on this problem until you feel more centered, more sure of what you have to offer.
Even when you’ve assessed your value and you think you’ve discovered the other people’s WIIFMs, influence may not seem to work. If that’s the case, perhaps it’s because the other people–the project manager or senior managers–may not have considered their value or what they want. When faced with this unevenness of perceived value, you may want to offer to help them define their value, and in the process help clarify their WIIFMs. There’s no simple solution when your colleagues don’t know what they want. Although we’d like to think we can figure out a way to work with everyone, sometimes you just can’t. Ultimately, you choose with whom you work.
As a test manager, one of your roles is to assess product state and report on it. You provide value by planning the product testing so that you can assess–along with the rest of the management team–the risk of shipping the product. The more widely you disseminate the data about the product state, the more perceived value you will provide, and the more influence you can have.
As an influential test manager, you can
- Ask vital questions about the product and project, and assess the answers to those questions
- Facilitate a discussion about tradeoffs for the release
- Generate and test the release criteria, so you have an objective way to answer the question “When is it ready to ship?”
You don’t have to feel out of control. The more you share your perception of the risks, the more you can influence what happens with your project, and the more successful you can make the organization you’ve invested in.
I thank the following reviewers for their helpful comments: Mark Druy, Elisabeth Hendrickson, Cem Kaner, Brian Lawrence, Brian Marick, Noel Nyman, Jeffery Payne, Dave Smith., Jerry Weinberg.
Cohen, Allen R. and David L. Bradford, Influence without Authority, John Wiley and Sons, New York, 1991.
Deming, Edward, Out of the Crisis, MIT Press, Cambridge, MA, 1986.
Gause, Don and Gerald M. Weinberg, Exploring Requirements, Quality Before Design, Dorset House, New York, 1989.
Kaner, Cem. Black Box Software Testing, a 3-day course taught by Cem Kaner, March 1999.
Marick, Brian. Classic Testing Mistakes. STAR 1997.
Rothman, Johanna and Brian Lawrence, A Pragmatic Strategy for NOT Testing in the Dark, Software Testing and Quality Engineering, Mar/Apr 1999.
Weinberg, Gerald M. Quality Software Management, Vol. 1 Systems Thinking, Dorset House, New York, 1992.
Like this article? See the other articles. Or, look at my workshops, so you can see how to use advice like this where you work.