How to Estimate Dependence in Moderate Dimensions Using Ranks and Sub-Sampling

• Activity Number: 35
• Type: Contributed
• Date/Time: Sunday, August 3, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Jerome Collet*+
• Companies: EDF R&D and MODAL'X, Paris X
• Keywords: sub-sampling ; U-statistics ; ranks ; regression

Abstract

It is well known that non-parametric methods suffer from the "curse of dimensionality". We propose here an estimation method, using sub-sampling and ranks, which seems not to suffer from this curse. In a n-sample of a d-dimensional random variable, we draw many m-sub-samples (m< n). For each sub-sample and each coordinate, one may compute the rank of an observation. So, for each sub-sample, we have m vectors of ranks, taking their values in {1, ..., m}^d. For each of the m^d possible values of these vectors, we count the sub-samples "reaching" this value. We first prove that this count is an estimation of the copula. We also prove that in the case of independence, for dimensions less than 5, the uncertainty of the estimated distribution increases almost linearly w.r.t. the dimension. Moreover, a simulation study shows that if we use this estimation to test independence, the number of observations needed to obtain a given power increases linearly with the dimension. We use this method to simulate cloud cover. First studies show we have to take account of dependence with season, temperature, and lagged values of the cloud cover, which makes 5 or 6 dimensions.