Why Do You Care About What “Everyone” Else Does?

Jurgen asked me to help publicize his survey. Ok, I’ve done it. Now, let me rant about explain why I think surveys like this are not useful, and may be harmful.

A survey does not take your context into account. Surveys about any practices without considering the industry, the products, and the management don’t tell you much. People lie to themselves about what they are really doing. And, they can harm efforts underway.

A senior manager saw a survey like this. He asked me what timeboxing was. I explained. Now he thought timeboxing was the best thing since sliced bread. He decreed that from now on, all projects would be timeboxed to no more than eight (8) weeks. The fact that some of the teams were using two week timeboxes and some were using straight kanban (not inside timeboxes) to finish valuable work was irrelevant to him. When I explained about the value of shorter timeboxes, he told me I was nuts. (I am, but not about that!) After all, he didn’t have any projects he could finish in 8 weeks, so how could anyone else? Not all senior managers are that clueless. But too many are.

How about the Scrum Master who, in response to a survey including questions about retrospectives, responded that they always did one. Their retrospectives lasted 20 minutes for a 4-week iteration, and consisted of a form: What went well (+), what wasn’t so hot (-), and what should change (Δ)? There were no action items, no adaptation based on reality. But he got a raise because he answered the survey in a way management wanted.

I don’t buy the survey or online assessments. You want to know what’s working for you? Learn how to lead a real retrospective. Then do one. (Or hire Esther or me :-) Get a real assessment.

These two examples are egregious examples of what can go wrong with surveys. They occur because there is no data to back them up. Does it matter if people exxagerate? Does it matter if management interprets something incorrectly and makes invalid assumptions? It doesn’t matter if you don’t care what everyone else does. But if you care, it matters. A lot.

As a very famous person said, What Do You Care What Other People Think? I would rather know what my context and reality are, what my particular constraints and issues are, and work in the best way I can to manage those constraints and issues.

rant off.

Jurgen, I may owe you, not the other way around!

About Johanna Rothman

I help managers and leaders do reasonable things that work.
This entry was posted in process and tagged , , , . Bookmark the permalink.

6 Responses to Why Do You Care About What “Everyone” Else Does?

  1. João Bernardino says:

    The problem isn’t in the survey, but in the way people use the results…

    If one’s gonna do something just because of a survey result, then they’re just behaving like a teenager (but everybody’s doin’ it, mom!), but that’s not the survey / surveyer’s fault.

    I find surveys sometimes interesting and sometimes useful, but we can’t follow the results blindly. They are many times good starting points for further research/study.

    In this particular case (Jurgen’s survey), and me being a wannabe agilist with no practical experience, I’m interested in knowing what practices people find useful.
    If after that I don’t research/study/analyze they’re usefulness/applicability in my case, it’s not really they survey’s fault is it?

    I do agree that there are limitations to the survey approach and results can sometimes be erroneous, misleading or misinterpreted but the problem isn’t in the tool, but the way people use it.

  2. João Bernardino says:

    I also think many times the usefulness / correctness of a survey is directly related to the way the survey is designed.

  3. You want to know what’s working for you? Learn how to lead a real retrospective.

    Thank you, Johanna!

    My favorite is companies that want to “be more agile” and so do this by picking one, single practice, in isolation so that they can say they are. “We hold daily scrums, so, look, we’re agile!” Never mind that that the “daily scrums” are hour-long status meetings on completely non-agile projects that serve no purpose other than insuring the folks on that project will fight the idea of adopting agile tooth and nail from here out.

    But then, why use common sense when you can just go off of a nice, neat little checklist? ;-)

  4. Jim Ward says:

    A survey is a tool, and just like any other tool, it can be used effectively or mis-used, even abused. Your examples point out abuses of the tool, such as answers on a survey being used for individual evaluations. However, there are very valid uses for surveys. Understanding these, and having them in your toolkit, can be an asset.

  5. Jack Vinson says:

    I haven’t looked at the survey in question, but one of the things that have bothered me about customer service surveys is their general lack of focus on … CUSTOMERS. Their focus is so often on “have we met our metrics.” Their metrics usually have nothing to do with whether they really helped me.

    Take the internal help desk survey. I recall a company where their reported numbers were always 99% or better. If they are that good, don’t bother surveying or find a better set of questions that give you room to improve.


  6. Liz says:

    Surveys and “Industry Standards” really cheese me. I had a boss once who always said – “Industry Standard is 1 QA resource for every x developers – why are we above that?” I kept asking “which industry?” as I was quite sure he didn’t have specs from any other high-stakes testing company. The term “Industry Standard” implies that you’re using metrics from companies that share an industry.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>