Assessing Intra- and Inter-Rater Device Reliability in a Rare Disease

• Activity Number: 687
• Type: Contributed
• Date/Time: Thursday, August 8, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Avital Cnaan*+ and Tina Duong and Fengming Hu
• Companies: Children's National Medical Center and Children's National Medical Center and Children's National Medical Center
• Keywords: intrarater reliability ; interrater reliability ; small sample size

Abstract

A study was designed to compare the intra- and inter-rater reliability of 2 devices used in assessing muscle strength in a rare neuromuscular disease in boys. The devices are different in structure (one is handheld and the other is anchored to a wall) and cost. While the handheld device costs substantially less, it is operator dependent, and subject to potentially a larger variability in measurements. Because the disease is rare, no single study site has a sufficient number of patients to assess reliability well, which requires adjustments to study design. This study was therefore designed to be conducted at 5 international sites, and each participating boy was evaluated 8 times, 4 sessions on each of 2 days within the same week. Four muscle groups were tested in each session, always in the same order. Eight testing sessions allowed for having each of 2 evaluators test each of the 2 devices on every participant twice, and thus allow assessing intra-rater reliability on both devices in addition to inter-rater reliability. Issues of randomization, evaluators, and potential testing fatigue of the patients were addressed. Surprisingly, the devices showed similar reliability results.