|
Abstract:
|
We study the situation where estimators built on a training set are combined using the empirical risk in a test set. In model selection, we examine when in the selected estimator has excess risk close to the optimal one, and to what extend this depends on a bound for (moments of) the initial estimators, and on the margin behavior (the latter being in particular of interest in classification problems). In linear or convex aggregation, the excess risk of the aggregated estimator is close to the optimal excess risk if aggregation is carried out over a limited number of initial estimators. When this number is very large however, a regularized empirical risk should be used. We study the Lasso penalty and prove oracle inequalities for general, possibly unbounded, loss functions.
|
