Ronaldo Iachan, PhD
ICF International, Calverton, Maryland
Dr. Iachan, a senior statistician, has 30 years of experience in statistical methods and applications, particularly in the areas of survey sampling design and analysis. For ICF, Dr. Iachan provides statistical expertise across divisions in projects in the areas of health, education, and social studies. He was a professor at Iowa State University and at the University of Wisconsin–Madison. He has served on ICF’s IRB for 12 years, and has been a statistical editor for the Journal of the American Medical Association for the past 15 years. He has extensive experience in statistical design and analysis, with more than 30 refereed articles published in statistical methods,. Dr. Iachan has extensive experience providing sampling and survey design support for many cancer-related projects for the Centers for Disease Control and Prevention (CDC) and other agencies, including cancer registry studies and heart disease prevention.
364 – Nonprobability/Web Sampling and Data Analysis
An Empirical Method to Establish Usability of Nonprobability Surveys for Inference
Robert D. Tortora
ICF International
Ronaldo Iachan, PhD
ICF International, Calverton, Maryland
Eric Miller
ICF
Non-probability surveys (NPS) are already widely in use in market research. However, their adoption for official statistics is much more problematic. AAPOR (2013) identifies a "Fit for Purpose" approach where the most difficult issue to address is making point estimates that are statistically valid, that is, can be used for statistical inference. This paper describes a methodology to empirically evaluate NPS surveys selected from a panel for statistical inference. The method compares estimates from an online panel with data from a gold standard probability survey. The key aspect of the methodology include transparency through an a priori decision rule motivated by the ASPIRE system developed by Bergdahl et al. (2014). We propose a distance metric and a predetermined cutoff value for deciding whether to accept or reject NPS estimates. Our decision rule is based on comparisons of 1) overall survey and subgroup estimates 2) the cv of the variability of the post-stratification weights and 3)the ratio of response rates. We illustrate our proposed empirical method by comparing data from a NPS quota sample for the Los Angeles area with a probability health survey of the same area.