When performing market research, the sample provider you choose is fundamental to the success and reliability of your research. To ensure the integrity of your data and eliminate potential fraud, consider asking the following questions to your sample provider:
Question 1: Are all respondents recruited directly, or is data collection outsourced to third parties?
Why It Matters:
Direct recruitment enables greater data quality controls. When respondents are recruited directly, it becomes easier to identify duplicates and bots, as the data collection process is closer to the researchers. This helps maintain the authenticity and accuracy of your survey responses. In its own work on detecting fraudulent respondents, Verasight analyzed two commercial opt-in panels and removed up to 13% of cases as fraudulent.
Question 2: How many surveys do respondents take per week?
Why It Matters:
Panels where respondents take a high number of surveys might introduce biases due to respondent learning effects, fatigue, or non-random drop-off. Research published in the Journal of Caring Sciences links over-surveying and fatigue to careless, lower-quality responses. Understanding the frequency of survey participation can help you assess the potential impact of these factors on your data quality. Verasight's own benchmarking of how often respondents take surveys found respondents at two large panels took far more per week. One panel averaged about 10 surveys, compared with roughly 3 for Verasight.
Question 3: Are respondents routed from one survey to the next survey?
Why It Matters:
Respondents being routed from one survey to the next may introduce a priming effect. If respondents have taken several surveys right before your survey, it can influence their responses or attention to detail. Pew Research Center documents these order effects, where earlier questions provide context that shapes later responses. Verasight's benchmarking of respondents routed from survey to survey found nearly 20% at two large panels took 6 or more surveys right before the current one. Just 4% of Verasight respondents did the same. Being aware of this routing process helps ensure the independence and unbiased nature of your survey data.
Question 4: Do respondents receive an incentive if they do not qualify for a survey?
Why It Matters:
If respondents only receive an incentive for qualifying, there may be an incentive for them to give dishonest answers to increase their chances of qualifying. By understanding the incentive structure, you can better assess the sincerity and reliability of respondents' answers. Verasight's research on how incentives shape respondent behavior explores this dynamic.
It's important to carefully evaluate and select sample providers who prioritize transparency, data quality, and ethical practices. By asking these key questions and understanding the implications, you can safeguard the reliability and integrity of your survey research, ensuring the data you collect is trustworthy and actionable.
