Statistical Inference for Big Data via Optimal Subsampling

• Activity Number: 453 - Novel Theory and Methods in Big Data Analytics
• Type: Invited
• Date/Time: Wednesday, August 1, 2018 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): HaiYing Wang*
• Companies: University of Connecticut
• Keywords: Logistic Regression; Massive Data; Optimal subsampling; Poisson sampling; Rare Events Data

Abstract

With limited computing resources compared with the exponentially growing data volume, subsampling-based methods have demonstrated pervasive potential in making better use of a fixed amount of computing power. In this talk, focusing on logistic regression, I will discus the problem of statistical inference for the Optimal Subsampling Method under the A-optimality Criterion (OSMAC). For OSMAC, the subsampling probabilities depend on unknown parameters, so a two-step adaptive algorithm is used. Consistency and asymptotic normality of the estimators from the two-step adaptive algorithm are established, and the asymptotic variance-covariance matrices for different estimators are compared. Extensive numerical results are also provided to demonstrate practical performance of the estimators based on OSMAC.