Impact of Persistent Type II Error on Data Quality in Periodic Survey Editing Methods

• Activity Number: 346 - Measuring and Improving the Quality of Government Statistics
• Type: Contributed
• Date/Time: Tuesday, August 1, 2017 : 10:30 AM to 12:20 PM
• Sponsor: Government Statistics Section
• Author(s): Brock Webb* and Chin-Fang Weng
• Companies: U.S. Census Bureau and U.S. Census Bureau
• Keywords: Manual survey editing ; Periodical survey data ; Type I error ; Type II error ; Data quality

Abstract

Manual survey editing to clean up response errors is important, but time consuming and costly. Therefore, only a partial list of the most likely errors is checked by survey statisticians during the data editing or data cleaning process. Thus, some errors are left undetected, that is Type II errors. When Type II error is present over time, it becomes a persistent feature of the data and we are unsure how closely it represents the truth. This study simulated response error using ten years of real stock market data to examine the longer-term impact of this persistent Type II error in editing periodical survey data. This study seeks to answer the question of whether the Persistent Type II error increases Type I errors (false positives) and Type II errors (false negatives), and does it effect the overall data quality over time?