Goodness-of-fit and Multilevel Models: Application to Survey Data from the Community Intervention Trial for Youth (CITY) Project

• Activity Number: 392
• Type: Topic Contributed
• Date/Time: Wednesday, August 10, 2005 : 10:30 AM to 12:20 PM
• Sponsor: Section on Survey Research Methods
• Author(s): DeMarc Hickson*+ and Lance A. Waller
• Companies: Emory University and Emory University
• Keywords: multi-level model ; goodness-of-fit ; time-space sampling ; Bayesian methods ; hard-to-reach population ; gay men

Abstract

CITY project personnel conducted time-space sample surveys for four consecutive summers of young men who have sex with men (MSM) in 13 communities throughout the United States as part of the evaluation of a community-level intervention. This survey approach introduced four levels to the hierarchical data structure: individual, venue, community, and time. Assumptions underlying criteria for comparing models within a classical modeling framework do not hold when comparing models that incorporate a hierarchical data structure (i.e., multilevel models) or models with improper prior distributions. The deviance information criterion (DIC) is a Bayesian model comparison criterion that combines both model fit and complexity and allows for the comparison of models with varying structure and dimension. A generalization of the coefficient of determination, R2 is a measure commonly used to compare the fit of a linear regression model and attempt to explain the variance at each level of a multilevel (hierarchical) model. We compare and contrast the performance of these measures of fit in our data.