4 Reasons Online Qual Consumer Studies Fail
Let’s admit it… many market researchers are either uncomfortable with online qual or they didn’t get the results they expected and shy away from it. That’s too bad, because the new wave of online qual tools and techniques are producing incredible insights for clients who have discovered the benefits of both technology, and adopting best-practices based on years of learning. Based on my experience over the last 15 years as an online qual researcher, I see four simple reasons why most online qual studies have not delivered on expectations. And all of them are fixable…with the right amount of experience and skills. Here is my take:
A Boring Activity/Discussion Guide
The discussion (or activity) guide needs to be clear, concise, and dynamic. Go beyond a battery of open-ended questions and use the variety of projective techniques that modern platforms offer. A study that goes beyond open-ends and mixes in respondent video, collages, perceptual maps, social activities, and storytelling will make things SO much more interesting to your respondents and your clients.Committee 2.
Committee Approach to Study Design
Avoid the committee approach where everyone gets to add in everything they could ever want to learn in one study. Don’t let it become a free-for-all. I’ve seen these more than a few times and you create a mountain of unstructured data loaded with redundancy and a lot irrelevancy detracting from your objectives. Not only will the data haul yield insufficient results it will bore your respondents. A key sign from respondents that the committee has left its mark is when you see comments from respondents like “I just answered that question…3 times!” Stick to your guns and assure clients that the insights will come out if the questions and projective exercises are well thought out and diverse.
Lack of Communication
I firmly believe that communication with respondents – from the beginning of the recruiting process through the completion of the study – is key. Be clear with respondents in the screener sharing exactly what the study is about, why they are important to the research, how much time is expected of them, how many days the study will take, and activities they are required to complete in what time frame. Moderators – send a morning note to everyone each day of the study giving them group encouragement and letting them know what they are doing on that particular day. Send at least one probe to all respondents on day 1 telling them, personally, how much you appreciate their contributions.
Insufficient incentives
Nothing will discourage a respondent more than doing a lot more work than they anticipated when recruited. Typically, a multi-day study should require a respondent to commit at least 30 minutes per day. If the study is interesting and well-designed, respondents will often spend a lot more time sharing because they want to, not because they have to. Typical incentives for online qualitative tend to be $100 for 3 days, $125 for 4, and so on. Store trips including video and/or pictures with added open- and closed-ends should include an additional $25+. Of course numbers can vary, but these are pretty tried and true guidelines. I have heard a few clients who tend to use $.50-$1 per minute that they expect the respondent to engage. I encourage higher incentives if the budget will allow.
After reading this you may think a few of these points are a blinding glimpse of the obvious. And they are. The lessons learned are pretty straightforward: Keep it simple. Pay attention to the basics of good research. Work with an online qual platform that is intuitive and user-friendly, and most importantly, is supported by seasoned consumer researchers. With a skilled team guiding you through the process, you should EXPECT better results with your next online study.
