|
We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, the minimax rate is determined to be (n log n)^{-s/(s+d)} + n^{-1/2} for compactly supported densities, where s \in (0,2] is the smoothness parameter and n is the number of independent samples. This result is then generalized to densities with unbounded support under Orlicz norm constraints. The optimal rate is achieved by an estimator based on certain polynomial approximation techniques, while the standard plug-in estimator with kernel density estimates is suboptimal.
One of the key steps in analyzing the bias relies on a novel application of the Hardy-Littlewood maximal inequality, which also leads to a new inequality on the Fisher information that may be of independent interest.
This is joint work with Yanjun Han, Jiantao Jiao, and Tsachy Weissman.
|
