Dirty and Unknown: Statistical Editing and Imputation in the SCF

• Activity Number: 572
• Type: Contributed
• Date/Time: Wednesday, August 6, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Arthur Kennickell*+
• Companies: Federal Reserve Board
• Keywords: Editing ; Imputation ; Nonsampling error ; Quality assurance

Abstract

Prevention of errors in surveys must always be the highest ideal, but in such a complex process as a survey there are limits on what is achievable, because of cost, the absence of strong instruments for control or the emergence of unforeseen outcomes. Thus, effort must be devoted to identifying errors, remediating them, and designing better means of prevention or limitation where that is possible. Editing is typically a key instrument of identification and remediation. However, editing can consume very substantial resources and because the outcome is unlikely to be perfect, the very act itself introduces additional risks to data quality. For these reasons, it has been argued (e.g., de Waal, 2013) that a selective approach to editing, focused as squarely as possible on the core analytical goal of a survey may be more appropriate than detailed review of all survey observations. For surveys supporting multiple uses, particularly ones involving multivariate analysis, there may be a need for a somewhat broader focus, but a more efficient approach may still be possible in such cases. This paper evaluates various approaches to selective editing, using various combinations of fully edited and unedited data from the 2010 Survey of Consumer Finances (SCF), a widely used survey covering household financial behavior and a variety of associated information. The paper also explores the potential importance of contamination of the imputation process under selective editing. While editing has its direct effect on individual data items, it also alters the set of information used in imputing the missing values that result from the unwillingness or inability of respondents to provide answers or from the resetting of values to missing during the editing process. The results of the paper support a selective approach to editing and they indicate that any resulting contamination of imputation is relatively minor in the case of the SCF.