A New Framework for Distance Metrics in High Dimension

• Activity Number: 634 - Recent Advancements in Distance and Kernel-Based Metrics and Related Learning Methods
• Type: Invited
• Date/Time: Thursday, August 1, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Xianyang Zhang* and Shubhadeep Chakraborty
• Companies: Texas A&M University and Texas A&M University
• Keywords: Distance covariance; Energy Distance; High Dimensionality; Independence Test; Two Sample Test

Abstract

We present new metrics to quantify and test for (i) the equality of distributions and (ii) the independence between two high-dimensional random vectors. In the first part of this work, we show that the energy distance based on the usual Euclidean distance cannot completely characterize the homogeneity of two high dimensional distributions. To overcome such a limitation, we propose a new class of metrics which inherit some nice properties of the energy distance in the low-dimensional setting and is capable of detecting the pairwise homogeneity of the low dimensional marginal distributions in the high dimensional setup. In the second part of this work, we propose new distance-based metrics to quantify the dependence between two high dimensional random vectors. In the growing dimensional case, we show that both the population and sample versions of the new metrics behave as an aggregation of the group-wise population and sample distance covariances. Thus it can quantify group-wise non-linear and non-monotone dependence between two high-dimensional random vectors.