High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: risk underestimation

DOI10.1214/10-AOS795zbMATH Open1274.62365arXiv1211.2917OpenAlexW2144809865MaRDI QIDQ620558FDOQ620558

Noureddine El Karoui

Publication date: 19 January 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We first study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the high-dimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate the parameters. The Markowitz problem in Finance is a subcase of our study. Assuming normality and independence of the observations we relate the efficient frontier computed empirically to the "true" efficient frontier. Our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix. In particular, the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1-p/n. Therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters. We also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities.

Full work available at URL: https://arxiv.org/abs/1211.2917

Recommendations

• On the realized risk of high-dimensional Markowitz portfolios
• The effect of estimation in high-dimensional portfolios
• Why estimation alone causes Markowitz portfolio selection to fail and what we might do about it
• Statistical inference for high-dimensional global minimum variance portfolios
• High dimensional minimum variance portfolio estimation under statistical factor models

zbMATH Keywords

high-dimensional inferenceconvex optimizationrandom matrix theoryelliptical distributionscovariance matricesconcentration of measureMarkowitz problemWishart matricesquadratic programsmultivariate statistical analysis

Mathematics Subject Classification ID

Asymptotic properties of nonparametric inference (62G20) Multivariate distribution of statistics (62H10) Quadratic programming (90C20) Estimation in multivariate analysis (62H12) Random matrices (probabilistic aspects) (60B20) Portfolio theory (91G10)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Asymptotic Statistics
• Title not available (Why is that?)
• Statistics on special manifolds
• Title not available (Why is that?)
• Title not available (Why is that?)
• Covariance regularization by thresholding
• A distribution-free M-estimator of multivariate scatter
• A well-conditioned estimator for large-dimensional covariance matrices
• On the distribution of the largest eigenvalue in principal components analysis
• Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices
• Sparse permutation invariant covariance estimation
• Regularized estimation of large covariance matrices
• Title not available (Why is that?)
• Local operator theory, random matrices and Banach spaces.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• The concentration of measure phenomenon
• Title not available (Why is that?)
• Title not available (Why is that?)
• Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices
• Title not available (Why is that?)
• Title not available (Why is that?)
• Concentration of measure and spectra of random matrices: applications to correlation matrices, elliptical distributions and beyond
• Enhancement of the applicability of Markowitz's portfolio optimization by utilizing random matrix theory
• Estimation for Markowitz Efficient Portfolios
• RANDOM MATRIX THEORY AND FINANCIAL CORRELATIONS
• The Distribution of the Sample Minimum-Variance Frontier
• Risk and asset allocation.
• The strong limits of random matrix spectra for sample matrices of independent elements
• Statistical models and methods for financial markets
• Noisy covariance matrices and portfolio optimization. II
• Title not available (Why is that?)

Cited In (34)

• On the Spectrum of Sample Covariance Matrices for Time Series
• Robustifying Markowitz
• Time-varying minimum variance portfolio
• On the inference about the spectral distribution of high-dimensional covariance matrix based on high-frequency noisy observations
• The Benefits of Diversification Between Bitcoin, Bonds, Equities and the US Dollar: A Matter of Portfolio Construction
• An eigenvalue distribution derived ‘Stability Measure’ for evaluating Minimum Variance portfolios
• On robust regression with high-dimensional predictors
• Asymptotics for high dimensional regression \(M\)-estimates: fixed design results
• High dimensional minimum variance portfolio estimation under statistical factor models
• CUSUM control charts for monitoring optimal portfolio weights
• High-dimensional Markowitz portfolio optimization problem: empirical comparison of covariance matrix estimators
• Statistical Inference for High-Dimensional Global Minimum Variance Portfolios
• Certifiably optimal sparse inverse covariance estimation
• A Nodewise Regression Approach to Estimating Large Portfolios
• Random matrix theory in statistics: a review
• Which bridge estimator is the best for variable selection?
• Using principal component analysis to estimate a high dimensional factor model with high-frequency data
• Network models to improve robot advisory portfolios
• On the Combination of Naive and Mean-Variance Portfolio Strategies
• Resolution of Degeneracy in Merton's Portfolio Problem
• The Dispersion Bias
• On the dimension effect of regularized linear discriminant analysis
• Sparse portfolio selection via Bayesian multiple testing
• Can we trust the bootstrap in high-dimension?
• Optimal Diversification in the Presence of Parameter Uncertainty for a Risk Averse Investor
• Estimation of the global minimum variance portfolio in high dimensions
• A nested factor model for non-linear dependencies in stock returns
• Cleaning large correlation matrices: tools from random matrix theory
• Optimal Shrinkage-Based Portfolio Selection in High Dimensions
• Effects of unlabeled data on classification error in normal discriminant analysis
• Risks of large portfolios
• On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators
• Non-parametric shrinkage mean estimation for quadratic loss functions with unknown covariance matrices
• Positive-Definite ℓ₁-Penalized Estimation of Large Covariance Matrices

Uses Software

• QRM

This page was built for publication: High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: risk underestimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q620558)