Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 88 - Data Visualization in Practice
Type: Contributed
Date/Time: Monday, August 9, 2021 : 10:00 AM to 11:50 AM
Sponsor: Section on Statistical Graphics
Abstract #318411
Title: Statistical Methods for Detecting Outlier Evaluators
Author(s): Molin Wang* and Yujie Wu and Bernard Rosner
Companies: Harvard T.H. Chan School of Public Health and Harvard T.H. Chan School of Public Health and Harvard Medical School/School of Public Health
Keywords: Quality control; outlier detection; false discovery rate; evaluator; reviewer

Epidemiologic and medical studies often rely on evaluators to obtain measurements. In this talk, we propose a two-stage method to detect ‘outlier’ evaluators whose evaluation results tend to be higher or lower than their counterparts. In the first stage, evaluators’ effects are obtained by fitting a no-intercept regression. In the second stage, hypothesis tests are performed to detect ‘outlier’ evaluators, where we consider both the power of each hypothesis test and the false discovery rate (FDR) among all tests. A proposed FDR vs. Power decision plot is fitted in the second stage, and based on this plot the evaluator-specific significance levels can be selected by achieving an acceptable tradeoff between FDR and power. Our extensive simulation study shows that our method not only has enough power for detecting true ‘outlier’ evaluators, but also is less likely to falsely reject true ‘normal’ evaluators. We illustrate our method by detecting ‘outlier’ audiologists in the data collection stage for the Conservation of Hearing Study, an epidemiologic study for examining risk factors of hearing loss in the Nurses’ Health Study II.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program