Effect Size Indices for Artificially Dichotomized Variables Measured with Error: An Empirical Investigation of Accuracy and Precision

• Activity Number: 552
• Type: Contributed
• Date/Time: Wednesday, August 7, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Jeffrey D. Kromrey and Isaac Li*+ and Patricia Rodriguez de Gil and Patrice Rasmussen and Jeanine Romano and Aarti Bellara and Harold Holmes and Yi-Hsin Chen and Rheta E. Lanehart and George MacDonald
• Companies: University of South Florida and University of South Florida and University of South Florida and University of South Florida and University of South Florida and University of South Florida and University of South Florida and University of South Florida and University of South Florida and University of South Florida
• Keywords: Effect sizes ; Simulation ; Reliability ; Statistical bias ; Dichotomy

Abstract

Monte Carlo methods were used to investigate the accuracy and precision of effect size indices in estimating what the standardized mean difference from a 2 X 2 sample table of dichotomized variables would have been had the data not been dichotomized. Normally distributed, continuous data were generated for two groups and the continuous variable was dichotomized at specified cut points. The factors manipulated in the simulation study included overall sample size (n1 + n2 = 30, 60, 120, 240), reliability levels (.5, .7, .8, .9, 1), population effect size (0, .2, .5, .8), continuous score cut point for dichotomization (.10, .25, .40, .50, .70), and population variance ratio (1:1, 1:2, 1:4). For each sample generated (100,000 replications), each of seven proposed effect size indices was calculated. Both the statistical bias and the RMSE were computed across the set of replications. Although the sample standardized mean difference became substantially biased in the presence of measurement error, the performance of the seven indices was not notably affected. Results were interpreted in terms of recommendations for estimating effect sizes with dichotomized variables.