The landscape of empirical risk for nonconvex losses

DOI10.1214/17-AOS1637zbMath1409.62117arXiv1607.06534MaRDI QIDQ1991675

Yanyan Li

Publication date: 30 October 2018

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1607.06534

zbMATH Keywords

uniform convergencenonconvex optimizationempirical risk minimizationlandscape

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Estimation in multivariate analysis (62H12) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Related Items (47)

On an extension of the promotion time cure model ⋮ A New Principle for Tuning-Free Huber Regression ⋮ Role of sparsity and structure in the optimization landscape of non-convex matrix sensing ⋮ A dynamic alternating direction of multipliers for nonconvex minimization with nonlinear functional equality constraints ⋮ Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Nonconvex Stochastic Optimization: Nonasymptotic Performance Bounds and Momentum-Based Acceleration ⋮ A stochastic subgradient method for distributionally robust non-convex and non-smooth learning ⋮ Graphical Convergence of Subgradients in Nonconvex Optimization and Learning ⋮ Robustness and Tractability for Non-convex M-estimators ⋮ Sharp global convergence guarantees for iterative nonconvex optimization with random data ⋮ Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model ⋮ Byzantine-robust distributed sparse learning for \(M\)-estimation ⋮ Implicit regularization in nonconvex statistical estimation: gradient descent converges linearly for phase retrieval, matrix completion, and blind deconvolution ⋮ Statistical Inference with Local Optima ⋮ Model-Assisted Uniformly Honest Inference for Optimal Treatment Regimes in High Dimension ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Finding second-order stationary points in constrained minimization: a feasible direction approach ⋮ Unnamed Item ⋮ Maximum likelihood for high-noise group orbit estimation and single-particle cryo-EM ⋮ Robust High-Dimensional Regression with Coefficient Thresholding and Its Application to Imaging Data Analysis ⋮ Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization ⋮ Adaptive and robust multi-task learning ⋮ Tractability from overparametrization: the example of the negative perceptron ⋮ Sample Size Estimates for Risk-Neutral Semilinear PDE-Constrained Optimization ⋮ Gradient complexity and non-stationary views of differentially private empirical risk minimization ⋮ A Path-Based Approach to Constrained Sparse Optimization ⋮ Uniqueness and stability for the solution of a nonlinear least squares problem ⋮ Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization ⋮ Discussion of: ``Nonparametric regression using deep neural networks with ReLU activation function ⋮ Finite-sample analysis of \(M\)-estimators using self-concordance ⋮ Oracle Inequalities for Local and Global Empirical Risk Minimizers ⋮ PUlasso: High-Dimensional Variable Selection With Presence-Only Data ⋮ Analysis of a two-layer neural network via displacement convexity ⋮ Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach ⋮ Linearized two-layers neural networks in high dimension ⋮ Statistical convergence of the EM algorithm on Gaussian mixture models ⋮ Iteratively reweighted \(\ell_1\)-penalized robust regression ⋮ A Well-Tempered Landscape for Non-convex Robust Subspace Recovery ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Universal statistics of Fisher information in deep neural networks: mean field approach^* ⋮ Unnamed Item ⋮ When do neural networks outperform kernel methods?* ⋮ Asymptotic Properties of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization ⋮ Batch policy learning in average reward Markov decision processes ⋮ Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in \(O(\sqrt{n})\) iterations

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Minimum distance Lasso for robust high-dimensional regression
• Gradient methods for minimizing composite functions
• High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
• Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer
• Multivariate statistical analysis. A high-dimensional approach
• A geometric analysis of phase retrieval
• Robust regression: Asymptotics, conjectures and Monte Carlo
• Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
• Simultaneous analysis of Lasso and Dantzig selector
• One-bit compressed sensing with non-Gaussian measurements
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• One-Bit Compressed Sensing by Linear Programming
• Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach
• Decoding by Linear Programming
• Robust Truncated Hinge Loss Support Vector Machines
• Distributed asynchronous deterministic and stochastic gradient optimization algorithms
• Morse Theory. (AM-51)
• Trust, But Verify: Fast and Accurate Signal Recovery From 1-Bit Compressive Measurements
• Regime Change: Bit-Depth Versus Measurement-Rate in Compressive Sensing
• High-dimensional estimation with geometric constraints: Table 1.
• Boolean Compressed Sensing and Noisy Group Testing
• Matrix Completion From a Few Entries
• A Stochastic Approximation Method
• Compressed sensing
• A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers

This page was built for publication: The landscape of empirical risk for nonconvex losses