Recently, a manager asked how he could detect critical thinking skills in candidates. I had to ask him more questions, so I could answer. Here's what he meant by critical thinking skills:

• The ability to think through a problem in a certain architectural domain
• The ability to deal with people across the organization in planning the next set of features and projects
• The ability to mentor others, choosing paths that would fit less-seasoned staff
• The abiltiy to discuss product strategy with people across the organization.

Those answers weren't what I expected when said "critical thinking skills." But they are. (I walked him through a partial job analysis until he could explain what he wanted.) And now, we were ready to develop some questions.

Here are some of the questions I suggested he consider and adapt:

• For the architectural problem thinking: "When was the last time you worked on this kind of architecture? ... How did you decide what to change? How did you change it?"
• For the cross-functional planning: "What was your role the most recent time your organization planned what to do when? ... How did that work?"
• For mentoring: "When did you last take on a mentoring role? ... What did you do?"
• Product strategy: "Has there been a time when you didn't agree with the product strategy? ... What did you do? ... Has there been a time when others had to be persuaded to your strategy? ... What did you do?"

These questions all start with closed questions to establish the candidate has done something like this, and then move onto behavior description questions to hear the story of the candidate has worked.

When the manager asked me about critical thinking skills, I thought he was going to talk about test creativity or debugging/development creativity. Without a quick job analysis, we would never have been speaking the same language. And, without analyzing what he really wanted, he would never have asked or adapted these questions for his candidates.

It's ok to say "critical thinking skills" as a shortcut when you're initially thinking about a job. But the context of those skills will help you define questions that will elicit what a candidate has done and how relevant that work is to you.

posted by Johanna Rothman at at 6:45 AM | link |