"Methodological Challenges for Handling Unmeasured Confounders in Causal Inference with Social Science Data

• Activity Number: 271 - Methodological Challenges for Handling Unmeasured Confounders in Causal Inference with Social Science Data
• Type: Invited
• Date/Time: Tuesday, August 1, 2017 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Reagan M Rose and Luke Miratrix* and Aaron Kaufman and Jason Anastasopoulos
• Companies: Harvard University and Harvard University and Harvard University and University of Georgia
• Keywords: Causal inference ; text analysis ; matching ; high-dimensional statistics

Abstract

In the world of text analysis there is little existing methodology for drawing causal conclusions when the covariates or outcomes are aspects of the text itself. We summarize the challenges for causal analysis in this domain and propose a general framework for estimating treatment effects in studies where the covariates and/or outcomes are summary measures built from text. First, we extend prior work on matching documents on features generated using text analysis methods. After matching, we estimate differential word use and sentiment using other text analysis tools. We demonstrate our procedure by comparing partisan bias across US news sources, as measured by their rates of coverage of issues and, given the same coverage, their different representation of topics. Here both the covariates (i.e., topics covered) and the outcome (i.e., language used and sentiment of covered content) are measured from the text. Our approach allows for investigation of two questions: "are news sources selecting content at different rates depending on their partisan alignment?", and furthermore, "when covering the same topics, are news sources presenting content using different language or sentiment?".