Cleaning large correlation matrices: tools from random matrix theory

DOI10.1016/j.physrep.2016.10.005zbMath1359.15031arXiv1610.08104OpenAlexW3104363895MaRDI QIDQ521794

Joël Bun, Jean-Philippe Bouchaud, Marc Potters

Publication date: 12 April 2017

Published in: Physics Reports (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1610.08104

zbMATH Keywords

spectral decompositionrandom matrix theorycorrelation matrixhigh dimensional statisticsrotational invariant estimator

Mathematics Subject Classification ID

Applications of statistics to actuarial sciences and financial mathematics (62P05) Nonparametric estimation (62G05) Random matrices (algebraic aspects) (15B52) Portfolio theory (91G10)

Related Items

Filtering time-dependent covariance matrices using time-independent eigenvalues, GOE statistics for Lévy matrices, Statistical inference for principal components of spiked covariance matrices, Limiting Spectral Distribution for Large Sample Covariance Matrices with Graph-Dependent Elements, Wigner and Wishart ensembles for sparse Vinberg models, Gradient descent dynamics and the jamming transition in infinite dimensions, Cluster structure in the correlation coefficient matrix can be characterized by abnormal eigenvalues, Network models to improve robot advisory portfolios, Improving portfolios global performance using a cleaned and robust covariance matrix estimate, The Dispersion Bias, PCA meets RG, Principal Eigenportfolios for U.S. Equities, Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising, Free dynamics of feature learning processes, CORRELATION MATRIX OF EQUI-CORRELATED NORMAL POPULATION: FLUCTUATION OF THE LARGEST EIGENVALUE, SCALING OF THE BULK EIGENVALUES, AND STOCK MARKET, Simulating realistic correlation matrices for financial applications: correlation matrices with the Perron–Frobenius property, Optimal cleaning for singular values of cross-covariance matrices, Dynamics of cluster structure in financial correlation matrix, An eigenvalue distribution derived ‘Stability Measure’ for evaluating Minimum Variance portfolios, Local laws for multiplication of random matrices, Spiked multiplicative random matrices and principal components, A memory-based method to select the number of relevant components in principal component analysis, Statistical diagonalization of a random biased Hamiltonian: the case of the eigenvectors, Spectra of large time-lagged correlation matrices from random matrix theory, A Review of Two Decades of Correlations, Hierarchies, Networks and Clustering in Financial Markets, Point process convergence for the off-diagonal entries of sample covariance matrices, A new spin on optimal portfolios and ecological equilibria, MIXANDMIX: numerical techniques for the computation of empirical spectral distributions of population mixtures, High dimensional deformed rectangular matrices with applications in matrix denoising, Subordination methods for free deconvolution, Eigenvector statistics of Lévy matrices, Generic features in the spectral decomposition of correlation matrices, Theq-dependent detrended cross-correlation analysis of stock market, Analyzing financial correlation matrix based on the eigenvector-eigenvalue identity, A cluster driven log-volatility factor model: a deepening on the source of the volatility clustering, Two short pieces around the Wigner problem, Random matrix models for datasets with fixed time horizons, Self-planting: digging holes in rough landscapes, Uncovering the dynamics of correlation structures relative to the collective market motion, Unnamed Item, Spiked sample covariance matrices with possibly multiple bulk components, Amalgamated free Lévy processes as limits of sample covariance matrices, Graph theoretical representations of equity indices and their centrality measures, Higher order moments of the estimated tangency portfolio weights, Estimation of high-dimensional dynamic conditional precision matrices with an application to forecast combination, Agglomerative likelihood clustering, Matrix Kesten recursion, inverse-Wishart ensemble and fermions in a Morse potential

Uses Software

• robustbase
• BayesDA
• SPECTRODE

Cites Work

• Invariant β-Wishart ensembles, crossover densities and asymptotic corrections to the Marčenko–Pastur law
• Theory of Financial Risk and Derivative Pricing
• Large Sample Covariance Matrices and High-Dimensional Data Analysis
• High‐Dimensional Covariance Estimation
• On the feasibility of portfolio optimization under expected shortfall
• Common risk factors in the returns on stocks and bonds
• Robust Statistics
• Some Applications of Matrix Derivatives in Multivariate Analysis
• Asymptotic Theory for Principal Component Analysis
• RELATIONS BETWEEN TWO SETS OF VARIATES
• Inequalities between the Two Kinds of Eigenvalues of a Linear Transformation
• Random Matrix Theory and Wireless Communications
• The elements of statistical learning. Data mining, inference, and prediction
• Ridge regression and asymptotic minimax estimation over spheres of growing dimension
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Large Sample Properties of Generalized Method of Moments Estimators
• A well-conditioned estimator for large-dimensional covariance matrices
• Around the circular law
• The singular values and vectors of low rank perturbations of large rectangular random matrices
• The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices
• Random matrices: universality of local eigenvalue statistics
• A Fourier view on the \(R\)-transform and related asymptotics of spherical integrals
• High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: risk underestimation
• Eigenvectors of some large sample covariance matrix ensembles
• Convergence rate of expected spectral distributions of large random matrices. I: Wigner matrices
• Second order statistics of robust estimators of scatter. Application to GLRT detection for elliptical signals
• The spectrum of heavy tailed random matrices
• No eigenvalues outside the support of the limiting empirical spectral distribution of a separable covariance matrix
• Large dimension forecasting models and random singular value spectra
• Multivariate analysis and Jacobi ensembles: largest eigenvalue, Tracy-Widom limits and rates of convergence
• Spectrum estimation for large dimensional covariance matrices using random matrix theory
• Universality results for the largest eigenvalues of some sample covariance matrix ensembles
• Concentration of measure and spectra of random matrices: applications to correlation matrices, elliptical distributions and beyond
• An orthogonally invariant minimax estimator of the covariance matrix of a multivariate normal population
• A distribution-free M-estimator of multivariate scatter
• On the eigenvectors of large dimensional sample covariance matrices
• An identity for the Wishart distribution with applications
• Empirical Bayes estimation of the multivariate normal covariance matrix
• The limiting empirical measure of multiple discriminant ratios
• Limit laws for random matrices and free products
• Wishart processes
• Multivariate empirical Bayes and estimation of covariance matrices
• Minimax estimators for a multinormal precision matrix
• Level-spacing distributions and the Airy kernel
• Multiplicative functions on the lattice of non-crossing partitions and free convolution
• Large deviations asymptotics for spherical integrals
• Limiting spectral distribution for a class of random matrices
• Planar diagrams
• On the distribution of the largest eigenvalue in principal components analysis
• Noisy covariance matrices and portfolio optimization. II
• On the empirical distribution of eigenvalues of a class of large dimensional random matrices
• Analysis of the limiting spectral distribution of large dimensional random matrices
• Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices
• Law of addition in random matrix theory
• On the large-\(N\) limit of the Itzykson-Zuber integral.
• Shape fluctuations and random matrices
• Independence test for high dimensional data based on regularized canonical correlation coefficients
• The random matrix regime of Maronna's M-estimator with elliptically distributed samples
• Spectrum estimation: a unified framework for covariance matrix estimation and PCA in large dimensions
• Subordination for the sum of two random matrices
• Asymptotics of unitary and orthogonal matrix integrals
• Random matrix theory in statistics: a review
• The Parisi formula
• Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices
• On the principal components of sample covariance matrices
• A Survey on the Eigenvalues Local Behavior of Large Complex Correlated Wishart Matrices
• Finite N corrections to the limiting distribution of the smallest eigenvalue of Wishart complex matrices
• Hidden noise structure and random matrix models of stock correlations
• On the Realized Risk of High-Dimensional Markowitz Portfolios
• Random Matrix Methods for Wireless Communications
• On the top eigenvalue of heavy-tailed random matrices
• Eigenvector dynamics under free addition
• Level Statistics and Localization Transitions of Lévy Matrices
• Rotational Invariant Estimator for General Noisy Matrices
• Universality of Wigner random matrices: a survey of recent results
• Free Probability Theory
• Large Deviations of Extreme Eigenvalues of Random Matrices
• Free Random Variables
• Beta ensembles, stochastic Airy spectrum, and a diffusion
• Differential Operators on a Semisimple Lie Algebra
• A simple derivation of the Tracy–Widom distribution of the maximal eigenvalue of a Gaussian unitary random matrix
• Top eigenvalue of a random matrix: large deviations and third order phase transition
• Large deviations of the maximum eigenvalue in Wishart random matrices
• Efficient computation of limit spectra of sample covariance matrices
• The large-Nlimit of matrix integrals over the orthogonal group
• Improved Estimation of Eigenvalues and Eigenvectors of Covariance Matrices Using Their Sample Estimates
• An Introduction to Random Matrices
• Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets
• A Brownian-Motion Model for the Eigenvalues of a Random Matrix
• The singular value decomposition: Its computation and some applications
• The eigenvalue spectrum of a large symmetric random matrix
• Asymptotic Statistics
• Convergence Rates of Spectral Distributions of Large Sample Covariance Matrices
• RANDOM MATRIX THEORY AND FINANCIAL CORRELATIONS
• Eigenvalue Estimation of Parameterized Covariance Matrices of Large Dimensional Data
• A nested factor model for non-linear dependencies in stock returns
• Dissecting financial markets: sectors and states
• An Intertemporal Capital Asset Pricing Model
• The planar approximation. II
• Replica field theory for deterministic models. II. A non-random spin glass with glassy behaviour