|
Abstract:
|
A model selection criterion is often formulated by constructing an estimator of a divergence measure. In the context of model selection, a divergence is a functional that reflects the disparity between two distributions: one corresponding to the fitted candidate model and the other corresponding to the generating model. Many divergence measures are asymmetric, meaning that an alternate measure is obtained by reversing the roles of the two distributions. As a consequence of this asymmetry, the divergence may differentially assess underspecified and overspecified models. Model selection criteria that estimate such measures may be excessively prone to favoring underfitted or overfitted models, in a manner that violates the philosophy of Occam's razor and the spirit of achieving an optimal bias/variance tradeoff. By symmetrizing such a divergence, and formulating model selection criteria based on the resulting measure, we can often obtain criteria that achieve a more desirable balance in terms of their selection patterns. We illustrate this general methodology through the development and investigation of variants of the Akaike information criterion and Mallow's Cp.
|
