Estimating High-Dimensional Covariance and Precision Matrices Under General Missing Dependence

• Activity Number: 173 - Recent Advances in Statistical Learning and Missing Data Handling
• Type: Contributed
• Date/Time: Tuesday, August 4, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Korean International Statistical Society
• Author(s): Seongoh Park* and Xinlei (Sherry) Wang and Johan Lim
• Companies: The Research Institute of Basic Sciences at Seoul National University and Southern Methodist University and Seoul National University
• Keywords: Convergence rate; covariance matrix; dependent missing structure; inverse probability weighting; missing data

Abstract

A sample covariance matrix S of completely observed data is the key statistic to initiate a large variety of multivariate statistical procedures, such as structured covariance/precision matrix estimation. However, the sample covariance matrix obtained from partially observed data is not adequate to use due to its biasedness. To correct the bias, an inverse probability weighting (IPW) estimator has been used in previous research. However, theoretical properties of the IPW estimator have been only established under very simple structure of missing pattern; every variable of each sample is independently subject to missing with equal probability.

We investigate the deviation of the IPW estimator when observations are partially observed under general missing dependency. We prove the optimal convergence rate Op({\log p / n}^{1/2}) of the IPW estimator based on the element-wise maximum norm. We also derive similar deviation results even when implicit assumptions (known mean and/or missing probability) are relaxed. In the simulation study, we discuss non-positive semi-definiteness of the IPW estimator and compare the estimator with imputation methods, which are practically important.