Let me pose a situation facing one of my clients now. Their business is expanding, and they've agreed to develop a product that's tangential to their current product line. This new product requires a new piece. If you're a software company, imagine it's a piece of hardware. If you're a hardware company, imagine it's software. If you're an instrumentation company, imagine you're bringing together some chemistry and physics into your already-existing biological analysis. Whatever it is, you're looking for someone new and different from any of the current people you have–and you, the hiring manager, don't know how to ask questions of candidates. What do you do?
Well, it's job analysis time. When you analyze this job, one of the biggest pieces of it will be how well the candidate explains his/her knowledge of his/her field, and how well that person can integrate his/her knowledge into the team. I'm not sure what else the analysis would tell you, but these seem to be the minimum for me.
Converting that analysis into an interview plan, I would start by asking the person to do a presentation to see how well this person explains their domain expertise. I would ask a bunch of behavior-description questions about problem-solving skills solo and within a team. And, I would ask for an extended audition, probably spending a half-day or a day with the project team, seeing how everyone's ideas intersect and how well people work together. (Be prepared to pay for a candidate's time here.) And I'd ask for references and check on how much in-depth knowledge this person has.It sure isn't easy. But it's not impossible. You can hire people when you don't know how to do their jobs. But spend time preparing, so you can ask good questions and assess the answers and any auditions.