|
Abstract:
|
Mutual information consistently has played important rule in many areas of statistics, machine learning and signal processing. There are various approaches to derive mutual information, approximately but exploring accurate estimation has been often a challenge, especially for high dimensional data when there is no parametric model for the data. This paper proposes Henze-Penrose mutual information as a new measure for higher order multivariate dependencies. We also introduce a novel algorithm, using Fridman-Rafsky statistics, to optimally achieve its estimator along with the convergence rate for certain classes of smooth functions. Among different extended versions, the advantage of having Henze-Penrose mutual information is that, we can guarantee an accurate estimator of dependency via direct graphical interpretation, without estimation or plug-in of densities. Despite the practical importance, we also show that Henze-Penrose mutual information can be used to provide improved bounds on R\'{e}nyi mutual information in terms of copula. We evaluate the theoretical results by designing a series of experiments and implementing the proposed algorithm.
|
