|
Abstract:
|
The theory of optimally weighted ensemble estimation is a general theory that can be applied to any ensemble of estimators to achieve improved convergence rates as long as the bias and variance of the base estimator can be expressed in a specific way. In particular, this theory has been applied to simple plug-in estimators of information theoretic quantities such as entropy, mutual information, and entropy to achieve the parametric rate when the densities are sufficiently smooth. However, the optimal weights for this ensemble estimation approach currently depend on knowing explicit expressions for the bias and variance. We empirically show that naively learning the weights by assuming the form of the bias and variance results in a bias reduction even when the bias differs from the assumed form, greatly expanding the potential field of applications. We also perform extensive experiments to demonstrate the robustness of the approach to the choice of tuning parameters.
|
