Utilizing Automated Statistical Edit Changes in Significance Editing

• Activity Number: 504
• Type: Topic Contributed
• Date/Time: Wednesday, August 3, 2011 : 10:30 AM to 12:20 PM
• Sponsor: Section on Government Statistics
• Author(s): Wendy Barboza*+ and Kay Turner
• Companies: National Agricultural Statistics Service and National Agricultural Statistics Service
• Keywords: significance editing ; statistical data editing ; selective editing ; outlier detection ; NASS

Abstract

The National Agricultural Statistics Service (NASS) is a statistical agency within the U.S. Department of Agriculture (USDA) that conducts hundreds of surveys every year and prepares reports covering virtually every facet of U.S. agriculture. NASS' traditional approach has been to perform a manual edit and review of all questionnaires for most surveys. As staff resources become more constrained, the agency has embraced technological advances. The goal of significance editing, defined as statistical data editing; selective editing; and outlier detection, is to (1) reduce the time and effort spent manually reviewing/correcting survey questionnaires, without damaging the quality of the resulting data, and (2) focus the manual effort on the accuracy of the survey questionnaires that strongly impact the overall results. During the survey process, the most influential records are identified by calculating a unit-level score based on the changes made by the automated statistical data edit. This paper provides details on these unit scores as well as the implementation of the significance editing concepts.