Savanta_37_online_sample_questions_US

37 QUESTIONS TO HELP BUYERS OF ONLINE SAMPLES
Data quality and validation
28. For work where you program, host, and deliver the survey data, what
processes do you have in place to reduce or eliminate undesired in-survey
behaviors, such as (a) random responding, (b) illogical or inconsistent
responding, (c) overuse of item non-response (e.g., “Don’t Know”) (d)
inaccurate or inconsistent responding, (e) incomplete responding, or (f)
too rapid survey completion?
Savanta utilizes carefully crafted trap questions in surveys to ensure that respondents do
not easily recognize them. We have found that incorporating open-ended trap questions,
along with other reliable and high-quality questions and checks, helps us maintain the
integrity of our survey data.
In order to maintain the overall quality of our projects, we conduct several general quality
checks, including:
• Open-ended question checks: Savanta employs automated checks to identify various
issues such as copied and pasted text, gibberish, obscenities, duplicate responses,
and the length of responses. We also have technology in place to translate open-ended
responses for surveys that are not in English, enabling us to run these automated
checks.
• Overuse of item non-response/straight-liners: We employ automated checks to
identify patterns in responses across specific question types. For example, we look for
instances where respondents repeatedly select the same positive or negative end of a
scale (e.g., “Strongly agree” or “Strongly disagree”), or consistently choose a particular
answer option (e.g., always selecting the first or last option in the answer list).
• Speeding check: Savanta identifies and removes respondents who rush through a
survey by assessing their completion time compared to the median question page
completion time. If a respondent spends 33% or less of the median completion time
on more than 40% of the survey pages, they are flagged and excluded from the final
dataset.
These quality checks help us ensure the reliability and validity of our survey responses.
savanta.com
14