Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data

• Activity Number: 509 - Recent Advances in High-Dimensional Time Series Analysis
• Type: Topic Contributed
• Date/Time: Thursday, August 11, 2022 : 8:30 AM to 10:20 AM
• Sponsor: Business and Economic Statistics Section
• Author(s): Ruey Tsay* and Zhaoxing Gao
• Companies: University of Chicgao and Zhejiang University
• Keywords: principal component analysis; high-dimensional time series; distributed computing

Abstract

This paper proposes a hierarchical approximate-factor approach to analyzing large-scale, high-dimensional heterogeneous time series data using distributed computing. The new method employs a multiple-fold dimension reduction procedure using Principal Component Analysis (PCA) and shows great promises for modeling large-scale data that cannot be stored nor analyzed by a single machine. Each computer at the basic level performs a PCA to extract common factors among the time series assigned to it and transfers those factors to one and only one node of the second level. Each 2nd-level computer collects the common factors from its subordinates and performs another PCA to select the 2nd-level common factors. The process is repeated until the central server is reached, which collects common factors from its direct subordinates and performs a final PCA to select the global common factors. The noise terms of the 2nd-level approximate factor model are the unique common factors of the 1st-level clusters. We focus on the case of 2 levels in our theoretical derivations, but the idea can easily be generalized to any finite number of hierarchies. We discuss some clustering met