Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data
From MaRDI portal
Abstract: This paper proposes a hierarchical approximate-factor approach to analyzing high-dimensional, large-scale heterogeneous time series data using distributed computing. The new method employs a multiple-fold dimension reduction procedure using Principal Component Analysis (PCA) and shows great promises for modeling large-scale data that cannot be stored nor analyzed by a single machine. Each computer at the basic level performs a PCA to extract common factors among the time series assigned to it and transfers those factors to one and only one node of the second level. Each 2nd-level computer collects the common factors from its subordinates and performs another PCA to select the 2nd-level common factors. This process is repeated until the central server is reached, which collects common factors from its direct subordinates and performs a final PCA to select the global common factors. The noise terms of the 2nd-level approximate factor model are the unique common factors of the 1st-level clusters. We focus on the case of 2 levels in our theoretical derivations, but the idea can easily be generalized to any finite number of hierarchies. We discuss some clustering methods when the group memberships are unknown and introduce a new diffusion index approach to forecasting. We further extend the analysis to unit-root nonstationary time series. Asymptotic properties of the proposed method are derived for the diverging dimension of the data in each computing unit and the sample size . We use both simulated data and real examples to assess the performance of the proposed method in finite samples, and compare our method with the commonly used ones in the literature concerning the forecastability of extracted factors.
Cites work
- A canonical analysis of multiple time series
- A massive data framework for M-estimators with cubic-rate
- A partially linear framework for massive heterogeneous data
- A robust procedure to build dynamic factor models with cluster structure
- A split-and-conquer approach for analysis of
- Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets
- Are more data always better for factor analysis?
- Asymptotic Properties of Least Squares Estimators of Cointegrating Vectors
- Banded spatio-temporal autoregressions
- Clustering time series by linear dependency
- Determining the Number of Factors in Approximate Factor Models
- Distributed clustering using collective principal component analysis
- Distributed estimation of principal eigenspaces
- Eigenvalue ratio test for the number of factors
- Estimating cross-section common stochastic trends in nonstationary panel data
- Factor modeling for high-dimensional time series: inference for the number of factors
- Forecasting Using Principal Components From a Large Number of Predictors
- scientific article; zbMATH DE number 4135256 (Why is no real title available?)
- Inferential Theory for Factor Models of Large Dimensions
- Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
- Model Selection and Estimation in Regression with Grouped Variables
- Modeling High-Dimensional Time Series: A Factor Model With Dynamically Dependent Factors and Diverging Eigenvalues
- Modelling multiple time series via common factors
- Multivariate time series analysis. With R and financial applications
- Nonstationary dynamic factor analysis
- Principal components estimation and identification of static factors
- Quantile regression under memory constraint
- The Generalized Dynamic Factor Model
Cited in
(5)- Selecting the number of factors in multi-variate time series
- Consistent model selection for factor-augmented regression within hierarchical factor structures
- Distributed loading estimation for general factor model
- Determination of the Effective Cointegration Rank in High-Dimensional Time-Series Predictive Regressions
- Discussion of: ``Independent component analysis by robust distance correlation: on robustness in dependence-based ICA
This page was built for publication: Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6185567)
