Thursday, 31 December 2015 00:46

The Most Common (And Dangerous) Market Research Mistake


While in college, psychologist Eleanor Longden sought help from a psychiatrist as she tried to deal with symptoms of schizophrenia. When one appointment ran very late, she excused herself, telling the doctor, “I’m reading the news at six.” The psychiatrist duly recorded that Ms. Longden had “delusions of being a television news broadcaster.”

In fact, Longden really did have to read the news on a student-run TV station, as she describes in Scientific American Mind. This story is both amusing and appalling, all the more so because one wouldn’t expect a trained psychiatrist to make assumptions or jump to unwarranted conclusions.

The gross misinterpretation by the psychiatrist is an example of confirmation bias. She was expecting to find evidence of mental illness in her patient, and as a result viewed the slightly improbable news-reading comment as a delusion. The psychiatrist made no effort to clarify the statement. One quick question to explore the topic, or even a shrink-talk, “Oh?” would have revealed the simple, factual explanation.

If a skilled psychiatrist, trained to look for small nuances in patient statements, can make this kind of egregious mistake, do you think marketers, could, too?

Indeed, we can and do make errors at least as bad as the one by Longden’s doctor. When we interpret customer feedback, or data from surveys and focus groups, it’s our natural tendency to interpret the data in a way that is consistent with what we believe ourselves. As the data rolls in, we want to blurt out, “I knew it!” or, “I told you so!” Rarely do we look for other ways of viewing the data, particularly explanations that might prove us wrong.


There are lots of ways that market research can go wrong, but this one is particularly insidious because it exploits a flaw in the way most of us think.

In Why So Much Market Research Sucks, I describe a brush with this phenomenon early in my marketing career. The CEO had achieved past success in selling commodity products at higher prices by differentiating with service and quality. He was convinced that the same strategy would work in our market, too. When a customer survey showed that “price” ranked behind multiple other factors in importance, nobody asked, “Why?” Nobody suggested that the question be asked in a different way, or that this surprising result be explored in more detail.

Instead, confirmation bias kicked in. The survey was viewed as proof that price wasn’t all that important, and that a differentiation strategy would allow at least a modest price premium over less skilled competitors.


The new strategy lasted just a few short weeks. Sales fell, and recovered only when prices were rolled back to competitive levels. In retrospect, price was critically important to our customers but was never an issue because everyone charged the same price. The survey results were accurate, but were taken to imply something that wasn’t true.

If you are doing market research or analyzing customer feedback, be very careful to look for alternate ways of interpreting the information. Look for dissenting voices within your team or company when you analyze customer data, and don’t be afraid to follow up on important data points. In my pricing example, there were a few experienced people who questioned the findings and strategy, but their objections were summarily dismissed as mere resistance to change.

In both of the examples above, just a little more exploration would have revealed that the initial assumptions were incorrect. But, since the statements seemed to confirm what was already thought to be true, nobody took that extra step.

In the coming years, Big Data, behavior metrics, and other quantitative measurements will give us more data than we’ve ever had before. This wealth of information won’t help the bottom line, though, if we use it to support our current beliefs instead of viewing it thoughtfully and seeking alternative ways to interpret it.

Source :




World's leading professional association of Internet Research Specialists - We deliver Knowledge, Education, Training, and Certification in the field of Professional Online Research. The AOFIRS is considered a major contributor in improving Web Search Skills and recognizes Online Research work as a full-time occupation for those that use the Internet as their primary source of information.

Get Exclusive Research Tips in Your Inbox

Receive Great tips via email, enter your email to Subscribe.