Microsimulation Models: Calibration, Validation, and Belief in Model Predictions

• Activity Number: 478
• Type: Invited
• Date/Time: Wednesday, August 6, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Carolyn M. Rutter*+
• Companies: Group Health Research Institute
• Keywords: prediction ; validation ; health policy ; goodness of fit

Abstract

Microsimulation models are an important tool for informing health policy decisions. Models provide a structure for combining a wide range of evidence that represents the current understanding of both disease and interventions to prevent or treat disease. This structure includes a description of heath states that describe key events in a disease processes and rules describing transitions between states. Parameter associated with transition rules are selected to achieve good fit to observed statistics through a process of model calibration. Once calibrated, models are used to predict population-level outcomes under different policy scenarios. Model validation, evaluation of model predictions for new data not used for calibration, is critical for developing confidence in model predictions but rarely occurs before models are used. This presentation focuses on issues related to microsimulation model calibration and validation, using a model for colorectal cancer screening as an example. Issues include the degree of complexity that captured by a simulation, the relative importance placed on different statistics, and the ability to summarize goodness of fit to multiple targets.