Online Program Home
My Program

Abstract Details

Activity Number: 340 - SPEED: Applications of Advanced Statistical Techniques in Complex Survey Data Analysis: Small Area Estimation, Propensity Scores, Multilevel Models, and More
Type: Contributed
Date/Time: Tuesday, July 31, 2018 : 10:30 AM to 12:20 PM
Sponsor: Survey Research Methods Section
Abstract #326973 Presentation
Title: The Problem of Analytic Error in Secondary Analysis of Survey Data: What We Know, and What We Need to Do About It
Author(s): Brady T. West* and Joe Sakshaug
Companies: University of Michigan and University of Manchester
Keywords: Analytic Error; Complex Sample Survey Data; Survey Data Analysis; Design-Based Inference; Total Survey Error

Recent research has suggested that analytic error needs to be a more prominent part of the Total Survey Error framework. We recently completed a three-year NSF-funded project exploring the prevalence and implications of analytic error in secondary analyses of complex sample survey data sets. To date, we have now reviewed and coded the analytic methods used in more than 250 research products (peer-reviewed journal articles, proceedings papers, conference presentations, and technical reports) presenting secondary analyses of complex sample survey data, and we have studied the inferential implications of using incorrect analytic approaches in both household and establishment survey data sets. We will discuss the main messages from this work: 1) secondary analysts appear to be performing incorrect analyses of large public-use survey data sets at an alarmingly high rate, potentially misleading the consumers of all forms of research (including peer-reviewed journal articles), and 2) the implications of performing incorrect analyses can be quite severe. We will conclude with recommendations for peer reviewers and data producers.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program