The Use of Indicators to Assess the Quality of Business Survey Returns During Data Collection

• Activity Number: 160
• Type: Topic Contributed
• Date/Time: Monday, August 4, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Survey Research Methods Section
• Author(s): Daniel Whitehead*+ and Yarissa González and Broderick Oliver
• Companies: U.S. Census Bureau and U.S. Census Bureau and U.S. Census Bureau
• Keywords: adaptive design ; data collection ; nonresponse bias ; representativeness ; response indicators ; response rate

Abstract

Data collection efforts often focus on maximizing survey response. However, increasing the response rate does not necessarily improve the quality of the estimates. For example, the respondents and nonrespondents might differ systematically on key survey characteristic(s). In this case, without successful data collection strategies to obtain data from underrepresented subpopulations, additional collection efforts may not improve the quality of the estimates. Using empirical data from two surveys, we examine two indicators, each measuring a separate property of the respondent sample: the R-indicator, which measures deviation from missing-completely-at-random; and the balance indicator, which measures the deviation of the respondent-based mean from the full sample mean for selected items. Examined in conjunction with the weighted volume response rate, which estimates the population coverage, these two indicators signal when the response set has stabilized but is not a random subset of the full sample. Thus, the current data collection strategy is at phase capacity, and new collection strategies -- targeted to specific subpopulations, are needed to achieve a balanced response.