Using Imperfect Fidelity Measures to Improve Statistical Inferences About Educational Interventions

• Activity Number: 193
• Type: Contributed
• Date/Time: Monday, August 2, 2010 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Xin Sun*+
• Companies: Southern Methodist University
• Keywords: imperfect fidelity ; effect size ; hieratical data ; optimal design ; measurement error

Abstract

In education research, imperfect fidelity (such as a new treatment or new method is not implemented as planed) has attracted more and more attention. Since imperfect fidelity will affect the outcome, ignoring the imperfect fidelity in analysis may lead to inaccurate statistical inference. We suggest a new estimate of effect size that takes imperfect fidelity into consideration in a hierarchical structure. We check the robustness of the estimate under different distributions of fidelity. We also investigate the optimal design of fidelity measure data collection when fidelity cannot be measured precisely.