|
Abstract:
|
We introduce the penalized Bregman divergence (BD) by replacing the negative log-likelihood in the conventional penalized-likelihood with a broad and important class of BD, which encompasses nearly all of the commonly used loss functions in the regression analysis, classification procedures and machine learning literature. We investigate new statistical properties of the class of penalized-BD estimators with a diverging number of parameters, and develop statistical inference tools. It is shown that the penalized-BD estimator, combined with appropriate penalties, achieves the same oracle property as the penalized-likelihood estimator, but asymptotically does not rely on the complete specification of the underlying distribution. Furthermore, the choice of loss functions in the penalized-BD classifiers has an asymptotically relatively negligible impact on classification performance.
|
