The Resurgence of Qualitative Research – Part II

Making Room for the Unexpected in Qualitative Research

Recently, I wrote about how qualitative research plays a role in market research that big data and social listening will have a difficult time replacing. This month, I’ll discuss how to best use Qualitative Research so that it helps generate new thinking that guides future plans versus big data results which, by their nature, focus on past behaviors.

How Not To Use Qualitative Research

I recently observed a series of focus groups for a client that insisted the moderator religiously follow the discussion guide. Every question in the guide needed to be asked in the order prescribed and, once a round of responses was gathered, the moderator was asked to move on to the next question without delay. The moderator wanted to explore certain responses at a deeper level and probe responses that didn’t appear to be immediately relevant to the question asked but was redirected by the client in between sessions until he acquiesced and read the discussion guide like a script. In the end, the client got their questions answered, but I feel a huge opportunity was lost. No room was left for the unexpected. The groups were not allowed to reveal anything that was outside the client’s frame of reference.

Tell Me What I Don’t Already Know

The problem with adhering so closely to a qualitative discussion guide is that it limits the possible results to those that fit within the client’s preconceived worldview. For example, if you’re exploring what customers want from their next auto purchase and have a battery (no pun intended) of questions about fuel efficiency, performance, quality and design, you’ll get back results that fit within your expectations, but the results will box you in to only consider results that you had preconceived going into the research session. I’m not advocating for a qualitative free-for-all but rather recommending leaving room for surprises and deeper insights.

Qual Is Not Quant

Another request that was recently made on a qualitative project was to record responses and tally them up in a spreadsheet to produce charts with percentages of respondents responding in certain ways. This is problematic for two reasons -- firstly, qualitative research rarely achieves statistically stable base sizes and secondly, responses to open ended qualitative questions have a lot of ‘noise’ in them that can be misleading without the interpretation of a qualified moderator. While it’s possible to ask open ended questions in quantitative research and accurately code the responses, the results are more stable because the quant environment is more controlled with no chance of answers being affected by responses given by others.

Qualitative Research Is Supposed To Make You Think

Quantitative research is designed to produce binary “yes/no”, “go/no-go” answers. Qualitative research is designed to make you think. The desire for all research to clearly reveal the answer of what to do next is understandable. It’s easier to make a decision if you have some research that unequivocally points to it. However, reading qualitative results as if they were quantitative is dangerous in that the results may not be representative of the larger population. Qualitative results are meant to reveal underlying attitudes and motivations that perhaps weren’t considered by the marketing team, but these findings cannot be taken at face value -- they need to be interpreted to determine how they relate to brand strategy.

Understanding the benefits and limitations of qualitative research is critical. Teams that properly interpret qualitative results are better able to take advantage of the power of qualitative research and get at deeper insights that lead to game-changing strategy decisions. Checking off boxes on a discussion guide yields safe, unexciting results that may help the company make it through the next sales quarter but will ultimately dead-end when fresh ideas are exhausted.