How sample sources can play a decisive role in the quality of data
Kirk Ward, EVP for TNS Marketing Science Americas, presented in May 2014 at the Market Research in the Mobile World North America 2014 event in Chicago. He talked about the impact of data sources, lengths of survey questionnaires, and consumer data collection devices on research results. He also posed a timely question about the quality of the data being gathered by market researchers, who leverage the popularity of smartphones and are continuously faced with the difficulty of recruiting respondents for online panels.
Ward’s presentation revealed both self-evident and startling conclusions. One, river sampling, which has developed an unsavory reputation over the years, should be used with caution. According to Ward, investigation needs to be done first in order to find out which river sampling sources would yield reliable results. It can be remembered that in 2008, Knowledge Networks’ Chief Statistician Charles DiSogra authored a study that detailed how and why river samples are not representative of anything. Second, the blending samples culled from different panels should be conducted with utmost care. The blending should be made consistent across multiple channels in order to reliably compare ensuing results over specific time periods.
Ward also showed that consumer survey results were more or less similar across a variety of devices such as PCs, smartphones, and tablets. The same is true for both non-optimized and optimized questionnaires, which are characterized by the use of conversational language. The only difference is that respondents tend to not complete non-optimized questionnaires. Meanwhile, longer questionnaires understandably generated much higher drop out rates among survey respondents.
Among Ward’s useful recommendations are as follows. Regardless of the data collection method being used, a mobile questionnaire length of less than six minutes can help improve completion rates. To lower early termination, optimized questionnaires should be used over non-optimized ones. All in all, Ward’s presentation successfully demonstrated how sample sources can play a decisive role in the quality of data being gathered, the choice of which data collection methodology to use, and the results derived from analyzing such data.