Group-Level Outcomes in Multilevel Designs: An Empirical Comparison of Analysis Strategies

• Activity Number: 441
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Social Statistics Section
• Author(s): Jeffrey Kromrey* and V. Lynn Foster-Johnson
• Companies: University of South Florida and Dartmouth College
• Keywords: simulation ; Type I error ; bias ; power ; multi-level

Abstract

Methods for analyzing multilevel data with group-level outcome variables were compared in a simulation study. The analytical methods included OLS analyses of group means, a two-step approach suggested by Croon and van Veldhoven (2007), and a Full Information Maximum Likelihood Latent variable technique proposed by Lüdtke et al. (2008). Type I error control, power, bias, standard errors, and RMSE in parameter estimates were compared across design conditions that included number of predictor variables, level of correlation between predictors, level of intraclass correlation, predictor reliability, effect size, and sample size. Results suggested that an OLS analysis of group means, with White's heteroscedasticity adjustment, provided more power for tests of group-level predictors but less power for tests of individual-level predictors. Further, this simple analysis avoided the extreme bias in parameter estimates and inadmissible solutions that were encountered with other strategies. Results were interpreted in terms of recommended analytical methods for applied researchers.