Reinforced Designs for Observational Studies of Treatment Effects: Multiple Instruments Plus Control Groups as Evidence Factors

• Activity Number: 335 - SRMS/SSS/GSS Student Paper Competition
• Type: Topic Contributed
• Date/Time: Tuesday, July 30, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Bikram Karmakar* and Dylan Small and Paul Rosenbaum
• Companies: University of Pennsylvania and University of Pennsylvania and University of Pennsylvania
• Keywords: causal inference; evidence factors; instrumental variables; sensitivity analysis; catholic schooling

Abstract

Absent randomization, inference about the effects caused by treatments depends upon assumptions that can be difficult or impossible to verify. Causal conclusions gain strength from a demonstration that they are insensitive to small or moderate violations of those assumptions, especially if that happens in each of several statistically independent analyses that depend upon very different assumptions; i.e., if several evidence factors concur. These issues often arise when one has several possible instruments, together with the option of a direct comparison of treated and control subjects. Does each purported instrument actually satisfy the stringent assumptions required of an instrument? Is a direct comparison without instruments biased by self-selection into treated and control groups? In this context, we develop a method for constructing evidence factors. We study the effectiveness of Catholic versus public high schools, constructing three evidence factors from three past strategies for studying this question. Although these analyses use the same data, we construct essentially independent tests that require very different assumptions and examine the degree to which they concur.