People Lie – So what do we do about it?

I just finished taking a survey sent to me by a local arts venue where I’ve attended events in the past. The survey asked me to indicate how often I go to cultural events and I noticed a strange urge in myself to overstate the number of events I go to in order to make myself feel more cultured than perhaps I really am. I believe I successfully fought that urge but as a consumer market researcher it’s worrisome to me to think that I’m probably like most people and usually want to say the right thing and highlight my positives while downplaying or downright ignoring perceived negatives.

I saw a similar phenomenon first hand a few years ago after conducting some focus groups for a Spanish language television commercial and later running the ad through quantitative testing.

In the focus groups the respondents, in this case Hispanic women, said the ad was too sexual in nature and offended them. In the quantitative research the ad score very highly on likability, recall sores and surprisingly high for persuasion. So were the focus group participants truly offended or were they too embarrassed to say they liked the ad? Did the anonymity of the quantitative setting allow for greater truthfulness? Or perhaps the ad was working on a subconscious level and those that were perhaps truly offended were still somehow drawn to the product when it came time to make a purchase decision.

As professionals who either gather or use consumer insights we have to address these questions when designing surveys and/or analyzing results. I personally like qualitative research to generate ideas and reveal aspects of the product or concept that perhaps we were overlooking but I take most comments related to likes/dislikes with a grain of salt. For those I prefer quantitative research and better-yet, indirect measures of like/dislike and purchase intent such as asking people to select products they would buy from a virtual shelf prior to viewing a creative clutter reel and having them do it again after exposure to the creative to see if the featured product was chosen the second time around.

Another well known aspect of Hispanic market research is a positivity bias. It can be very frustrating to hear that your Hispanic consumers love everything about your product when sales are, in-fact, dipping. In order to counteract that we generally utilize a forced choice methodology when comparing the desirability of product features. Simply asking how much they like a feature on 5 or 10 point scale will yield a lot of “5’s” and “10’s” in your data without much discrimination among the answers. Branching can also be used to reduce overly positive answers by first asking if they agree or disagree with an attribute or statement followed by asking the level of agreement. The issue with both these methods is that they add length to a survey and thus limit the number of questions/features one can include per survey but the improved quality of the end-data makes them well worth considering.

Ultimately we have to trust that consumer research participants are being honest with their responses (after pulling out straight liners, speeders, etc.) The trick is to determine when consumers themselves are unaware of their own biases and design questions that uncover real-world behaviors that consumers themselves may not even be fully conscious of. So next time you’re asked how often you watch educational or cultural programming on television, eat healthy or go to the theater ask yourself if your thinking of your actual self or that ideal you that lives in your imagination.

–
Roy Eduardo Kokoyachuk
ThinkNow Research