Online Program

Identifying Best Practices for Inter-Laboratory Comparisons and Between-Method Studies (309897)

Keywords: Random-effects models, Bayesian, heterogeneity

A key task in measurement science is to combine independent measurements made by different scientists, at different times, or in different places. Random-effects models (REMs) combine these measurement results in a way that recognizes explicitly the contributions that different sources (i.e. within-study uncertainties and between-study differences) make to the overall uncertainty. The extensive literature in the field and variety of estimation methods available suggest that quantifying between-study variability is essential for the effective combination of these measurements. The principal question that scientists wishing to use these methods ask is: which statistical model and data reduction procedure should I use? In this work l compare the coverage properties of heterogeneity variance estimates obtained using many different methods, but in contrast to previous comparisons I include Bayesian methods with weakly informative prior distributions and explore new models (e.g. Laplace REMs). The different models and methods are compared using a variety of simulated and real datasets that vary in size and in the extent of their adherence to the assumptions of the usual REM.