Managing Quality on a Large Qualitative Research Study with Complex Respondent Recruitment Criteria
Julie Feldman
RTI International
Sarah K. Heimel
U.S. Census Bureau
Katherine Kenward
Research Support Services
M. Mandy Sha
RTI International
Qualitative methods, such as cognitive interviewing, are often used to pretest questionnaires. Commonly observed limitations include a small number of respondents, purposively chosen recruitment criteria, and interviewer variability. Managing quality associated with these operational aspects is particularly important to the success of large qualitative research studies with complex recruitment criteria. In 2010-2011, Census Bureau conducted a large-scale cognitive testing study of the Targeted Follow-up (TCFU) questionnaire, a detailed instrument designed to resolve instances of people who appear to have been duplicated in the Census. This study faced the challenge of completing 226 cognitive and 50 ethnographic interviews over a total of 6 months and two rounds, using a team of 13 highly-qualified interviewers in five states and 7 recruiters experienced with cold-calling. The majority of the interviews were conducted in the field with actual 2010 US Census participants recruited from 27 living situations (both households and Group Quarters), without the respondents knowing that they were duplicated. Stringent requirements were put in place to protect respondent confidentiality and privacy mandated by Title 13. This paper presents a description of the methods to manage quality and estimates their impact. We measure quality by the success of representing a diversity of living situations and duplications, the quality of the TCFU information collected, and interviewer and recruiter productivity. Our efforts to manage quality centered on making continuous improvement on recruitment strategy-setting and monitoring, interviewer selection and training, and communications and logistics management.
