Estimation of High-Dimensional Covariance Matrices with Incomplete Data

• Activity Number: 311
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 8:30 AM to 10:20 AM
• Sponsor: IMS
• Author(s): Anru Zhang* and Tony Cai
• Companies: University of Pennsylvania and University of Pennsylvania
• Keywords: adaptive thresholding ; bandable covariance matrix ; extended sample covariance matrix ; missing data ; optimal rate of convergence ; sparse covariance matrix

Abstract

Missing data occurs frequently in a wide range of applications. In this paper, we consider estimation of high-dimensional covariance matrices in the presence of missing observations under a general missing completely at random model in the sense that the missingness is not dependent on the values of the data.

Based on incomplete data , estimators for sparse, bandable and spiked covariance matrices are proposed. Both theoretical and numerical properties of the estimators are investigated. Minimax rates of convergence are established under the spectral norm loss and the proposed estimators are shown to be rate-optimal under mild regularity conditions. Simulation studies demonstrate that the estimators perform well numerically. The methods are also illustrated through an application to data from four ovarian cancer studies. The key technical tools developed in this paper are of independent interest and potentially useful for a range of related problems in high-dimensional statistical inference with missing data.