Interviewer Variability and Survey Estimates: Quantifying Data Quality

• Activity Number: 152
• Type: Contributed
• Date/Time: Monday, July 30, 2007 : 10:30 AM to 12:20 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Novie Younger*+ and Damion Francis and Shelly McFarlane and Jan van den Broeck and Georgiana Gordon-Strachan and Andriene Grant and Deanna Ashley and Rainford Wilks
• Companies: The University of the West Indies at Mona, Jamaica and University of the West Indies and University of the West Indies and University of the West Indies and University of the West Indies and Ministry of Health, Jamaica and Ministry of Health, Jamaica and University of the West Indies
• Keywords: Intra-observer correlation coefficient ; observer design effects ; observers ; random effects models ; precision ; means

Abstract

Ensuring data quality in national health surveys using large numbers of observers is important. Assessment of data quality and validity of survey estimates is desirable. Training and certification contribute to improved data quality but maintenance of quality in the field is often not assessed. Health survey data can quantify observers' influence on precision of sample means. Random effects models estimate association between observers' and supervisors' ("gold standard") independent replicate measurements adjusted for time lag. We estimate the intra-observer correlation coefficients (IOCC), observer design effects and the impact of observer variability on the variance of means. These results quantify survey data quality and suggest optimal intervals between interviewers' measurements and their "gold standard" for the purpose of quality control.