Confidence Intervals and F-Test for Intraclass Correction Coefficients Based on Three-Way Mixed Effect Models

• Activity Number: 300
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Education
• Author(s): Hong Zhou*+ and Paige Muellerleile and Debra Ingram and Seok Wong
• Companies: Arkansas State University and Marshall University and Arkansas State University and University of Memphis
• Keywords: Evaluation ; generalizability theory ; reliability ; statistics

Abstract

Intraclass correlation coefficients (ICCs) are commonly used in behavioral measurement and psychometrics. The formulas for deriving ICCs, or generalizability coefficients, vary depending on which models are specified. This article gives the equations for computing confidence intervals and conducting F tests for three-way mixed models with one fixed facet. The authors also provide the formulas for finding the sample size when the researcher wishes to maintain a minimum desired generalizability coefficient. Finally, the authors illustrate the analytic differences among four models using one hypothetical data set and then use real data from student course evaluations to further illustrate one of the four models.