High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
From MaRDI portal
Publication:693741
Abstract: Although the standard formulations of prediction problems involve fully-observed and noiseless data drawn in an i.i.d. manner, many applications involve noisy and/or missing data, possibly involving dependence, as well. We study these issues in the context of high-dimensional sparse linear regression, and propose novel estimators for the cases of noisy, missing and/or dependent data. Many standard approaches to noisy or missing data, such as those using the EM algorithm, lead to optimization problems that are inherently nonconvex, and it is difficult to establish theoretical guarantees on practical algorithms. While our approach also involves optimizing nonconvex programs, we are able to both analyze the statistical error associated with any global optimum, and more surprisingly, to prove that a simple algorithm based on projected gradient descent will converge in polynomial time to a small neighborhood of the set of all global minimizers. On the statistical side, we provide nonasymptotic bounds that hold with high probability for the cases of noisy, missing and/or dependent data. On the computational side, we prove that under the same types of conditions required for statistical consistency, the projected gradient descent algorithm is guaranteed to converge at a geometric rate to a near-global minimizer. We illustrate these theoretical predictions with simulations, showing close agreement with the predicted scalings.
Recommendations
- Non-convex projected gradient descent for generalized low-rank tensor regression
- Nonconvex Low-Rank Tensor Completion from Noisy Data
- Fast global convergence of gradient methods for high-dimensional statistical recovery
- Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization
- Implicit regularization in nonconvex statistical estimation: gradient descent converges linearly for phase retrieval, matrix completion, and blind deconvolution
Cites work
- scientific article; zbMATH DE number 4088699 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 918103 (Why is no real title available?)
- A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
- Atomic Decomposition by Basis Pursuit
- Covariate Selection for Linear Errors-in-Variables Regression Models
- Fast global convergence of gradient methods for high-dimensional statistical recovery
- High dimensional inverse covariance matrix estimation via linear programming
- High-dimensional graphs and variable selection with the Lasso
- High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
- Improved matrix uncertainty selector
- Lasso-type recovery of sparse representations for high-dimensional data
- Missing values: sparse inverse covariance estimation and an extension to sparse regression
- Multiplicative Errors-in-Variables Models with Applications to Recent Data Released by the U.S. Department of Energy
- On the conditions used to prove oracle results for the Lasso
- Polynomial Regression and Estimating Functions in the Presence of Multiplicative Measurement Error
- Reconstruction From Anisotropic Random Measurements
- Restricted eigenvalue properties for correlated Gaussian designs
- Simultaneous analysis of Lasso and Dantzig selector
- Sparse permutation invariant covariance estimation
- Sparse recovery under matrix uncertainty
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
Cited in
(only showing first 100 items - show all)- Penalised robust estimators for sparse and high-dimensional linear models
- The generalized equivalence of regularization and min-max robustification in linear mixed models
- Weighted \(\ell_1\)-penalized corrected quantile regression for high dimensional measurement error models
- A Unified Framework for Change Point Detection in High-Dimensional Linear Models
- On two recent nonconvex penalties for regularization in machine learning
- An ensemble learning method for variable selection: application to high-dimensional data and missing values
- Subspace estimation from unbalanced and incomplete data matrices: \({\ell_{2,\infty}}\) statistical guarantees
- Lasso guarantees for \(\beta \)-mixing heavy-tailed time series
- Regularized estimation and testing for high-dimensional multi-block vector-autoregressive models
- Estimation from nonlinear observations via convex programming with application to bilinear regression
- Rejoinder of ``Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
- A tight bound of hard thresholding
- Oracle inequalities for local and global empirical risk minimizers
- On Robustness of Principal Component Regression
- On consistency and sparsity for high-dimensional functional time series with application to autoregressions
- Confidence sets in sparse regression
- High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
- High-dimensional regression with potential prior information on variable importance
- Estimation of high-dimensional graphical models using regularized score matching
- Scalable interpretable learning for multi-response error-in-variables regression
- Lasso estimation for spherical autoregressive processes
- Penalized Estimation and Forecasting of Multiple Subject Intensive Longitudinal Data
- Sparse estimation via lower-order penalty optimization methods in high-dimensional linear regression
- Sparse linear models and \(l_1\)-regularized 2SLS with high-dimensional endogenous regressors and instruments
- Robust subspace clustering
- Inference in high dimensional linear measurement error models
- \(\ell_{0}\)-penalized maximum likelihood for sparse directed acyclic graphs
- Poisson Regression With Error Corrupted High Dimensional Features
- Finite sample theory for high-dimensional functional/scalar time series with applications
- Sparse principal component analysis with missing observations
- Inference for high dimensional linear models with error-in-variables
- Minimax rate-optimal estimation of high-dimensional covariance matrices with incomplete data
- Optimal detection of sparse principal components in high dimension
- On the uniform convergence of empirical norms and inner products, with application to causal inference
- Learning partial differential equations for biological transport models from noisy spatio-temporal data
- Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error
- Regularized estimation in sparse high-dimensional time series models
- Non-convex projected gradient descent for generalized low-rank tensor regression
- Going beyond oracle property: selection consistency and uniqueness of local solution of the generalized linear model
- An efficient two step algorithm for high dimensional change point regression models without grid search
- Optimal sparse linear prediction for block-missing multi-modality data without imputation
- Performance bounds for parameter estimates of high-dimensional linear models with correlated errors
- The landscape of empirical risk for nonconvex losses
- Fast global convergence of gradient methods for high-dimensional statistical recovery
- Balanced estimation for high-dimensional measurement error models
- Regularized estimation of high-dimensional factor-augmented vector autoregressive (FAVAR) models
- Robust inference of risks of large portfolios
- An \(\{\ell_{1},\ell_{2},\ell_{\infty}\}\)-regularization approach to high-dimensional errors-in-variables models
- On parameter estimation for high dimensional errors-in-variables models
- Scale calibration for high-dimensional robust regression
- Estimating high-dimensional covariance and precision matrices under general missing dependence
- scientific article; zbMATH DE number 7625194 (Why is no real title available?)
- Minimax Optimal Procedures for Locally Private Estimation
- Regularized estimation of high‐dimensional vector autoregressions with weakly dependent innovations
- On higher order isotropy conditions and lower bounds for sparse quadratic forms
- Structure estimation for discrete graphical models: generalized covariance matrices and their inverses
- Greedy algorithms for prediction
- Lower bounds for finding stationary points I
- Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates
- scientific article; zbMATH DE number 7306869 (Why is no real title available?)
- Kernel Ordinary Differential Equations
- Kernel Knockoffs Selection for Nonparametric Additive Models
- Multiple Change Point Detection in Reduced Rank High Dimensional Vector Autoregressive Models
- Estimating the Covariance of Fragmented and Other Related Types of Functional Data
- Inference in Approximately Sparse Correlated Random Effects Probit Models With Panel Data
- MEBoost: variable selection in the presence of measurement error
- Multi-Task Learning with High-Dimensional Noisy Images
- STRATOS guidance document on measurement error and misclassification of variables in observational epidemiology. II: More complex methods of adjustment and advanced topics
- scientific article; zbMATH DE number 7750672 (Why is no real title available?)
- Are Latent Factor Regression and Sparse Regression Adequate?
- Nonparametric covariance estimation for mixed longitudinal studies, with applications in midlife women's health
- Low-rank matrix estimation via nonconvex optimization methods in multi-response errors-in-variables regression
- Screening Methods for Linear Errors-in-Variables Models in High Dimensions
- D4R: doubly robust reduced rank regression in high dimension
- Model selection in high-dimensional noisy data: a simulation study
- Weighted l1‐Penalized Corrected Quantile Regression for High‐Dimensional Temporally Dependent Measurement Errors
- Linear regression with partially mismatched data: local search with theoretical guarantees
- Inference for heteroskedastic PCA with missing data
- UNIFORM-IN-SUBMODEL BOUNDS FOR LINEAR REGRESSION IN A MODEL-FREE FRAMEWORK
- Optimal nonparametric testing of missing completely at random and its connections to compatibility
- Sparse Identification and Estimation of Large-Scale Vector AutoRegressive Moving Averages
- Asymptotic Properties of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization
- Sparse estimation in high-dimensional linear errors-in-variables regression via a covariate relaxation method
- Calibrated zero-norm regularized LS estimator for high-dimensional error-in-variables regression
- Concentration of measure bounds for matrix-variate data with missing values
- An improved algorithm for high-dimensional continuous threshold expectile model with variance heterogeneity
- Adaptive Bayesian SLOPE: Model Selection With Incomplete Data
- Nonparametric classification with missing data
- The EAS approach for graphical selection consistency in vector autoregression models
- Norm statement considered harmful: comment on `Evolution of unconditional dispersal in periodic environments'
- Preserving privacy between features in distributed estimation
- Analysis of global and local optima of regularized quantile regression in high dimensions: a subgradient approach
- A Note on Cross-Validation for Lasso Under Measurement Errors
- High-dimensional data segmentation in regression settings permitting temporal dependence and non-Gaussianity
- FNETS: Factor-Adjusted Network Estimation and Forecasting for High-Dimensional Time Series
- L 0 -regularization for high-dimensional regression with corrupted data
- A Sparse Learning Approach to Relative-Volatility-Managed Portfolio Selection
- Bayesian vector heterogeneous autoregressive modelling
- Double bias correction for high-dimensional sparse additive hazards regression with covariate measurement errors
This page was built for publication: High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q693741)
