An Evaluation of Confidence Intervals for Composite Reliability Coefficients

• Activity Number: 81
• Type: Contributed
• Date/Time: Sunday, July 31, 2011 : 4:00 PM to 5:50 PM
• Sponsor: Social Statistics Section
• Author(s): Ken Kelley*+ and Sunthud Pornprasertmanit
• Companies: University of Notre Dame and University of Kansas
• Keywords: Reliability ; Composite Scores ; Confidence Intervals ; Coefficient alpha ; Coefficient omega

Abstract

Composite scores are the sum of multiple components (e.g., tests or questionnaires) and are widely used in many fields when the components measure the same underlying construct. Whenever there is interest in a composite score, the reliability of the composite, specifically the internal-consistency, should be evaluated. Depending on the underlying structure of the components, there are different reliability coefficients that can be used (e.g., alpha or omega). Coefficient alpha is the most widely used measure of reliability for composite scores, but it is more restrictive in its assumptions than omega. Regardless of the exact coefficient selected, the estimated coefficient almost certainly differs from the corresponding population value. Thus, it is important to provide a confidence interval for the population coefficient so as to not over-interpret the point estimate. However, multiple methods of confidence interval formation exist and it is not clear which is most effective. Multiple confidence interval approaches are examined and recommendations are made. Additionally, the methods discussed have been implemented in the MBESS R package so that methods can be easily implemented.