A Novel Approach for Evaluating Alerting Methods in Biosurveillance

• Activity Number: 160
• Type: Topic Contributed
• Date/Time: Monday, August 2, 2010 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistics in Defense and National Security
• Author(s): Jian Xing and Soyoun Park and Howard Burkom*+
• Companies: CDC and SRA International and The Johns Hopkins University
• Keywords: biosurveillance ; disease surveillance ; kappa statistic ; CUSUM

Abstract

Obstacles to evaluation of alerting algorithms for disease surveillance have been the variety of background data types and the lack of accepted outbreak signatures. The objective of these algorithms is to alert human epidemiologists for investigation/response decisions. With a static labor force, health monitors face a growing workload with increasingly complex data streams. As an alternative to ROC-like test measures, an approved, anonymized time series collection containing known outbreaks from the National Notifiable Diseases Surveillance System data history was presented to a group of epidemiologists for selection of desired alert patterns. Two analyses will be presented: the agreement among respondents using matrices of kappa statistics by event and overall, and the variation among algorithms in detection performance according to individual participant response and by consensus.