Online Research. Why? Why Not?

Debbie Davis, Director of Market Research, BMO, Toronto
Presented to the CARF Seminar - Market Research Surveys: Online Versus Traditional – Jan. 18, 2008

Debbie Davis reminded the seminar that online is only one possible methodology in the researcher’s toolbox. She said that while online research is not the Holy Grail, neither is more traditional research. Which methodology is best depends upon the product being researched and the target audience.

Davis said that the reality is that online research is no longer a fad but is here to stay. She points out that in some cases the use of online research is almost a given – the evaluation of a website, customer satisfaction studies where e-mail address are available and subjects have given permission to be contacted, or where the evaluation of visual material is under study. She said that online research generally is cheaper by about 20% than more traditional research methodology and can take about half the time to provide data than telephone methods. While the quality of responses to some open-ended questions is better with online than with telephone methodology, she also warns that online research offers the respondent the opportunity to give negative or even rude feedback that is not always welcomed by the client.

She said that some of the considerations of whether to use online research include the connectedness of the target audience. While telephone penetration is almost universal, some target groups have the same level of Internet as telephone penetration. She also pointed out that if your customer contacts you by Internet or interfaces with your company by Internet, then using the web as a medium to survey your customers is valid. The availability of a suitable sample must be considered when deciding between online and telephone studies. Some narrowly-defined target groups which are difficult to find using RDD methods may be available through an online panel for example.

While some advertisers are reluctant to switch some historical advertising tracking studies to online methodology, it was her experience that online research gave very similar results to telephone studies when parallel tests of the two methods were conducted over a 9-month period. Generally speaking, the online research provided more positive data than telephone – but not always. The online tracking study tests results had either lower costs or larger samples, faster turnaround times, and were superior for tracking awareness to radio, TV and out-of-home advertising. The online tracking study tests also yielded data that when compared to telephone-based research would have drawn the same conclusions or the same business decisions would have been made

However, Davis warns, persons considering using online research should be aware that the methodology is not the answer to all market research questions and that there are times when traditional methods are preferable. She also urges that careful examination of the online panel should be conducted before committing to the process. Online panels can be good and bad – with over-used or over-researched panels or panels with professional respondents – just like telephone panels. She also reminds research users that online data simply will be different from telephone-generated data so one cannot simply switch to online research without parallel testing.