Online Program

Knowing who they are looking for: Effects of revealing the target population on coverage rates and response rates in household surveys.

*Frauke Kreuter, University of Maryland & IAB/LMU 
Stephanie Eckman, Institute for Employment Research (IAB), Germany 
Roger Tourangeau, Westat 

Keywords: screening, eligibility, coverage, nonresponse, interviewer effects

Many surveys include screening interviews to identify members of the eligible population or members of rare subgroups slated for oversampling. Failure to find members of these groups drives up survey costs and may introduce bias into the estimates. There is evidence that members of the target population are often undercovered in surveys that screen for them, but are covered more fully in surveys that do not screen for them. One of the best documented instances of such a screening shortfall occurred in the National Longitudinal Survey of Youth, 1997 Cohort, with a coverage ratio of only about 70 percent of the targeted age group compared to 90 percent for all other age groups. Such screening shortfalls could reflect respondent motivation to screen out rather than refuse the survey interview. However, this form of underreporting could also reflect interviewer motivations, since field interviewers are often evaluated based on their nonreponse rates but not on the eligibility rates they achieve. This paper reports results from an experiment that varied three factors believed to affect coverage rates: The amount of information given about the target population in the advanced letters, and the format of the screener itself, with full household roasters being tested against screening questions that explicitly asked about the target population. In addition, interviewers were assigned to different payment schemes; interviewers got either a monetary incentive for completing a full interview or for completing just the screener. Our design allows the estimation of main effects and interactions. We found strong effects in eligibility rates and response rates as a function of the screener format, indicating that respondents do hide their eligibility when providing with information about the target population of interest. Interviewer payment schemes showed effects on productivity during refusal conversion, but not on eligibility or coverage.