Why Puzzles and Riddles Discriminate

At last week’s Agile 2006 conference, I led a tutorial called “Hiring for an Agile Team.” I made a statement that some of the participants challenged:

Using puzzles and riddles discriminate against anyone who isn’t a (middle-upper class) white American suburban male.

(I’d forgotten the middle-upper class part when I was leading the session.) So, what everyone wants to know is: Where is my data?

First, let me explain why I said this. (This post is a bit of a rant.)

  • Girls, for example, do not have access or the alone time to spend doing books of puzzles and riddles. It is socially unacceptable for even the geekiest girl you know to do this. A girl who spends time pursuing puzzles and riddles for her own pleasure runs the colossal risk of being ostracized from all the other girls. Boys tend to discover puzzles and riddles during middle school and continue to pursue them through high school. Middle and high school for girls is much more about social ability and social connections.
  • The people who can afford to buy the puzzle and riddle books (to practice them and become better at them) are middle-to-upper economic class.
  • Puzzles and riddles appeal to a limited number of personality types–types frequently found in high-tech jobs. In particular, they appeal most often to NTs, especially INTJ and INTP types. (NTs are visionaries in Do Your Interview Questions Discriminate For or Against Your Needs?

So, the people who practice with puzzles and riddles are the people who have the disposable income, time, and social acceptability to do so. I don’t know enough about middle class high school boys from other cultures, but I see this all the time in the US.

If you’d like to see some other information, take a look at How Would You Move Mount Fuji?. There are some hints at the articles at Career Resource Center but no reference. I also wrote Brainteaser Interviews Showcase Lack of Interviewer Skill, not Candidate Expertise. And, take a look at Brainteasers Inappropriate for Job Interviews by John Kador, who literally wrote the book.

So, have I convinced you yet? Maybe not. Maybe you still think you need a way to know if this candidate is smart enough for you, or you don’t feel competent to start a conversation with something like a puzzle or a riddle to dissect.

If you need to know whether a person is smart enough, use behavior-description questions and auditions. Don’t be afraid to ask the “What did you learn from that experience” question as a follow up from your behavior-description questions. If you need people who are smarter than the people we normally have in our field, use the job analysis to understand why. (I bet you don’t, but that’s another rant.)

If you need a conversation starter, I have some ideas on building rapport already in this blog, but I’ll write another piece about building rapport. Tongue-in-cheek: Really, if you’d been socializing in high school instead of using puzzles and riddles to avoid social interactions, you might already know how to do this.

Joel discusses ways to really interview in The Guerrilla Guide to Interviewing and he discusses auditions in The Perils of JavaSchools. So, if you’re looking for real discrimination data, sorry, I don’t have it. I only have my common sense.

Remember, the goal of an interview is to evaluate how well a candidate will do at work, specifically in your workplace. Puzzles and riddles may evaluate a certain kind of personality, and possibly even raw IQ. But they won’t tell you how well a candidate can work with your group or on your products. And that’s the point of an interview.

18 Replies to “Why Puzzles and Riddles Discriminate”

  1. Because I’ve seen some of those boys succeed at social situations later. Clearly, I do not have any experience being a boy in middle/high school. If it’s just as bad for boys, that’s pretty awful.

  2. What makes you think that a boy who spends time pursuing puzzles and riddles for his own pleasure does *not* run the colossal risk of being ostracized from all the other boys (and more importantly for boys, from girls)?

  3. Johanna – you have an interesting take on the this subject. I’m insulted that your generalization about puzzles in interviews doesn’t take into account Canadians 🙂
    My experience with giving people puzzles and/or brainteasers and/or random programs to test (I treat, rightly or wrongly, these activities as all requiring a similar mind set – not necessarily skill set) in an interview is it allows me to see how they react to the unknown. That’s what testing is like in our context. We often get products that do things we have no idea about – but the key is all the testers that work at PlateSpin know where to find the right Oracles.
    Some of the observations I make during a puzzle, brain teaser, testing question:
    Do they give up right off the bat?
    Do they ask questions or sit silent pondering?
    Do they make different attempts or approaches?
    What areas (if any) do they get stuck on?
    Can they explain what they are thinking?
    If yes – great. If no – what’s the reason?
    There are others but this is a good starter list.
    The way your post comes across is that you’ve tried puzzles in interviews and didn’t find them helpful for the context with which were interviewing for. Would that be a fair generalization?

  4. Johanna —
    I’d add that raw IQ doesn’t predict job success either.
    Goleman and other cite studies that show that (given that a certain level of cognitive ability is required for entry into the field) social abilities a greater contributor to success.

  5. So, I was in the session that Johanna mentions (Hi Johanna) and was very interested in this claim about puzzle questions. I was particularly excited by the inference that I drew (perhaps mistakenly) that there was some external evidence to support my own belief that they are bogus–well, this isn’t quite that although it is good for me that someone as experienced as Johanna shares my preference against these tests.
    What I’ve found is that puzzle questions discriminate against me and pretty much everyone else I know that I think of as a skilled developer of software systems. Sometimes in ways more subtle than that we just don’t know the “right” answer, and there usually is one. Very few of these people are middle-upper class white American suburban males.
    There was a brief exchange with one of the other folks at the session that brought the issue home to me. This fellow (who seemed strongly upper-middle etc to me) claimed that puzzle questions expose “how the candidate thinks” and gave the example question “how would the world be different if snow were black?” My response: what if I grew up in a village in southern India and I don’t know what snow is?
    I’ve worked with people with that background, and their first winter in northern England was a shock to them. Water spontaneously freezing in any form was completely alien to their whole life’s experience. The fellow at the session insited that even so I _would_ know what snow is and would be able to tackle the question. He was quite forceful about it. I couldn’t think of a way to explain the gulf of understanding exposed there.
    What occured to me later, though, was that the “how does the candidate” think model seems reminiscent of the notion that a good trainer can train all things equally well, a good manager manage all things equally well. It seems as if there is a belief that a “good thinker” can think about all things equally well–and that these good thinkers are always the best folks to have on your team. This would seem to be the exact opposite message to that to be taken from Myers-Briggs, Belbin and so forth.
    It had occured to me before that there was something wrong with that model, but I caouldn’t quite pin down what–but now I have the idea that the allegedly universal “good thinking” style is really the style of the middle-upper class white American suburban male, and that’s a useful new perspective. Thanks.

  6. Totally agree (and I’m a white, upper middle class formally suburban mail).
    Puzzles have the appearance of abstract “how you think” tests. But based on my experience with crosswords (and with logic puzzles to a lesser extent), I find that the more I do them, the better I get.
    For example, when I do the NY Times crossword regularly, I find that I can master Thursday’s puzzle after a month or so. Does that mean that my thinking has changed in the last month? No – it means that I’ve learned how to perform a certain task. Same thing with Sudoku.
    I believe that if you take people with the same work aptitude and told one to solve two logic puzzles a day for three months, that person would do very well on a puzzle interview but wouldn’t necessarily do better at their job.

  7. Ok, but why make public a controversial statement that is a blatant stereotype? You can argue your point all you want, but the fact is it would take me less than 5 minutes to find someone that makes your argument look silly. Are you trying to stir up controversy? I’ve read your book and found it very insightful, however, I don’t see the need for this comment. Even if this statement were to be true 80% of the time, you run the risk of offending a large number of people. Why bring race, income, and sex into the argument?

  8. I strongly disagree with your comments – and not just because I happen to love puzzles.
    It seems to me that there are two different issues here. One is whether the puzzle interview is a useful tool in assessing a potential employee. I agree with Adam White’s comments above that it certainly can be, depending on what type of position you’re filling. If it’s a job that requires strong problem-solving ability, then it would be highly useful to see how a candidate approaches solving a problem, particularly if the problem requires similar reasoning ability as one he or she would be likely to encounter on the job. The fact that this tool has been misused by interviewers who are only interested in the right answer or showing off their own knowledge – or who pick inappropriate puzzles – doesn’t negate that potential usefulness. Furthermore, assuming that you are correct that puzzles only appeal to a certain personality type, that said personality type is prevalent amongst people who have high-tech jobs, and that the real goal of an interview is not to find out whether a person can do the job but whether he or she can fit into your team, then it seems clear to me that your team will most likely have a high number of people with that personality type already, so the candidate who does well in that type of interview should fit in just fine.
    The second issue, more disconcerting to me, is whether using a puzzle is discriminatory, particularly against candidates who are not “middle-upper class white American suburban males.” I find it interesting that you haven’t presented any evidence – even “common sense” – that puzzles discriminate against urban dwellers or non-white people, and you admit that it may not be a purely American phenomenon. So really what you’re saying is that puzzles discriminate against people who grew up poor, because they couldn’t afford puzzle books, and against women.
    The first statement is ridiculous. Everyone in America, including people of lower income, spends money on entertainment. Some people go to movies, some buy video games. Puzzle magazines, by comparison, are much less expensive, and there are many ways to get puzzles for free (http://www.PuzzleMonster.com, for example). Whether someone chooses to spend money and free time on puzzles is another issue – but it’s certainly not something that only wealthy people do.
    That leaves us with your assertion that puzzles discriminate against women, because it is “socially unacceptable” for girls to spend time on puzzles in high school. It is also socially unacceptable for girls to spend time developing the technical skills that would suit them well in a high-tech field; does that mean it is discrimination for interviewers to expect female applicants to have those skills?
    I suspect that the only discrimination here is your own, against puzzles and people who enjoy them. You imply that one of the primary reasons interviewers use them is that they were the kids in high school who avoided social interactions – i.e., losers. But someone of your intelligence should know that name-calling is simply no way to make a case or win an argument.
    And that’s MY rant. 🙂

  9. Happened across this thread… thought I’d chime in.
    Puzzles not only can provide insight into a candidate’s thought process and attitude, but they can also be a valuable tool in broadening the problem solving abilities of those who are already on staff. Puzzles can help with brainstorming tasks, can educate, and encourage lateral thinking both visually and literally.
    Discriminatory? No way! If a candidate runs from a little puzzle or riddle challenge, what else will they run from? Even if they are not puzzlers, they should be eager to solve the problem before them, and welcome the opportunity to show their stuff.
    I’ll also submit to you that a high school drop-out who was reared as an avid and diverse puzzler and a hands-on kind of guy could very well challenge many college grads at the IQ table. The upper middle class white guy stereotype of puzzlers is flat wrong. I see people who do puzzles as people who enjoy figuring things out, period. This applies whether they succeed or fail (though they prefer success) and it is the process that stimulates them as they look forward to the win. By the way, this same process has been shown to prolong the clarity of mind in elderly folks. It’s the use-it-or-lose-it thing.
    Ted, puzzle maker, http://qfold.com

  10. I find this thread very interesting from a couple of perspectives: I’m African-American, and I have trouble with puzzle/logic interview questions. I have a history of similar types of trouble, such as the old analytical GRE section, and some classes I took as an undergraduate and grad student.
    I may come back to this thread later, but one point I think that bears mention is the knowledge of the importance of puzzles to the modern software professional is not widely known outside of the field. It certainly wasn’t known to me (or my parents) when I was in grade school. I actually used to solve lots of puzzles back then: anagrams, logic diagrams, mazes, rearranging numbers in movable tiles so they were in order, etc. However, I had no idea that what I was doing was going to be “important” for my future career (in fact, I didn’t know at the time that I was going to do CS — granted, I’m talking about the late 1960s and 1970s here). So neither I nor my parents made an issue of a regular, comprehensive approach to problem solving through puzzles. I’d be just as likely on any given day to read a book, draw a map, play an imaginary game, play solitaire, or do something else that didn’t quite exercise the intellect in the way that puzzles do. As time went on, puzzles became less important as I had other things to do, such as study algebra or trig (which are not puzzle-based), or work during summers (while in high school) to help pay for my future education. I was a lifeguard, so I had to learn things like water safety and first aid, plus exercise (swimming, running), which took up a lot of time.

  11. Obviously, your post is very biased. As long as you can’t support your claims with data, they aren’t of any value apart from that they represent your very own opinion but nothing more.
    I have an opinion too (but admittedly no data either). Being from a different cultural background (I’m a bloody European 😉 ) all I can say is that regarding the social acceptance of geeks/nerds, there’s hardly any difference between the genders. So by discouraging the use of riddles in job interviews you don’t fight for equal chances for women and men but rather just discriminate geeks and nerds (female or male). Why? Às they don’t get the opportunity in the interview to show where they are good at. Instead, if they don’t have it, they need to “fake” social competence (or what the general perception of social competence is – social competence in dealing with “virtual” people usually doesn’t count despite of the fact that we’re more and more interacting with people we only know through the Internet). Like this, you either select for truly socially competent people or at least emotionally intelligent social dorks who are able to fake it well enough. But you leave out those emotionally not so gifted social dorks who are brilliant masterminds.
    Having quite some experience in the IT industry myself I know there are lots of jobs for the latter type of people. These are exactly the jobs a socially competent person would probably quit in short time due to the lack of real-life interactivity with other people. A possibly introverted genius would probably excel in the same job, however.
    Conclusion: Don’t over-simplify and don’t ban riddles from being used in interviews just because you don’t like them. Use them where it makes sense, don’t use them where it doesn’t make sense. As with many other interview techniques.
    NB: Please note that I don’t claim geeks/nerds being necessarily socially less competent than non-geeks/non-nerds. Maybe they tend to be. My point is that the interview should be targeted towards skills the job position really demands and give everybody (including geeks/nerds) a fair chance to show off where they’re good in instead of having to fake skills they don’t have just to convince the interviewer and get the job. Besides, we probably agree that in an interview, faking social competence is much easier than faking a high IQ. And interview/assessment centers are mostly a matter of faking things (well enough).
    PS: If you wondered: Yes, my native language isn’t English – I beg your pardon for all the mistakes.

  12. This ia a great webblog! I found this site in search for an answer to why I’m being dicriminated at work. I’m an African American, women and IT professional. I felt my supervisor was testing my IQ by bringing in puzzles for me to solve. At first it was a challenge and co-workers would find humor in seeing me struggle. I’ve been able to solve these puzzles with very little or no help on several occassions. After reading the materials here, I understand how helpful puzzles can be and use it to my advantage. My co-workers however, still look for opportunities to prove I’m inferior.

  13. I’m good at puzzles and riddles but only because I started playing chess in my early 20s as a way to cope with chronic pain less emotionally.

    Gender and racial stereotypes harm us all. By your logic, men are discriminated against if we expect them to have empathy for others because they are raised not to show emotion in most environments. You don’t need to prove that our current hiring methods are biased because look around you! How many jobs have you worked where there are overwhelmingly white men with maybe a few Asian/Indian people (mostly men) in lower prestige positions? If there is another woman in a tech team with me, she is almost always white and still not a developer. Not true at my current job but nearly all former jobs.

    People want to see themselves as fair & will do anything to protect the illusion that if there is a basis problem, it certainly isn’t with them! Look at the results.

    1. HI Lanette, I agree that stereotypes harm us all. It’s quite difficult to create an unbiased interview, yet it’s worth our time to do so. Your experience bears that out.

      We have trouble as humans being unbiased. It’s worth trying to see our biases and creating situations that don’t reinforce those biases.

  14. Johanna,
    I agree with you about puzzles and the like during interviews. I think they really don’t show how a person would examine and pull apart a system under test. Also, most of the times the puzzles are of the type that most people will look at and go “Huh?!”. They serve no purpose (IMO) other than to see how a person would react to a ridiculous situation. They are just like the question of “Tell me about your weaknesses and how you got around them”. That doesn’t tell me how the person is going to test a system, and/or they just B.S. around it themselves.

    Admittedly I have given a quiz about a piece of code to debug for automation or given an example of an ambiguous requirement and asked a candidate how they would improve upon it. But those are real world situations to show how you would solve a problem.

    Finally, yes… I do suck at those stupid puzzles. But I’m going for a job as a Tester, not as a stupid question/puzzle solver.

    1. Jim, asking questions about a piece of code that is either part of your system or similar to your system is a terrific audition. Great idea.

      We solve problems at work, not necessarily puzzles. The thing that bugs me the most about puzzles is that there is One Clever Answer. If you develop a different answer than what the interviewer expects, too often the interviewer thinks you have the Wrong Answer.

Leave a Reply