Abstract:
|
In logistic regression, separation occurs when a linear combination of the predictors can perfectly classify part or all of the observations in the sample, and as a result, finite maximum likelihood estimates of the regression coefficients do not exist. Gelman et al. (2008) recommended independent Cauchy distributions as default priors for the regression coefficients in logistic regression, even in the case of separation, and reported posterior modes in their analyses. As the mean does not exist for the Cauchy prior, a natural question is whether the posterior means of the regression coefficients exist under separation. We prove two theorems that provide necessary and sufficient conditions for the existence of posterior means under independent Cauchy priors for the logit link. We demonstrate empirically that even when the posterior means exist under separation, the magnitude of the posterior samples for Cauchy priors may be unusually large. In our examples of datasets with separation, normal priors led to reasonable scales for the posterior draws of the regression parameters, and comparable or even better predictive performance than other priors.
|