Sharp oracle inequalities for least squares estimators in shape restricted regression

DOI10.1214/17-AOS1566MaRDI QIDQ1750286zbMATH OpenOpenAlexWikidataFDO

Authors Pierre C. Bellec

Publication date 18 May 2018

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1510.08029

zbMATH Keywords

concentrationconvexityGaussian widthminimax ratesshape restricted regression

Mathematics Subject Classification ID

Minimax procedures in statistical decision theory (62C20) Nonparametric regression and quantile regression (62G08)

Abstract: The performance of Least Squares (LS) estimators is studied in isotonic, unimodal and convex regression. Our results have the form of sharp oracle inequalities that account for the model misspecification error. In isotonic and unimodal regression, the LS estimator achieves the nonparametric rate $n^{-2/3}$ as well as a parametric rate of order $k/n$ up to logarithmic factors, where $k$ is the number of constant pieces of the true parameter. In univariate convex regression, the LS estimator satisfies an adaptive risk bound of order $q/n$ up to logarithmic factors, where $q$ is the number of affine pieces of the true regression function. This adaptive risk bound holds for any design points. While Guntuboyina and Sen (2013) established that the nonparametric rate of convex regression is of order $n^{-4/5}$ for equispaced design points, we show that the nonparametric rate of convex regression can be as slow as $n^{-2/3}$ for some worst-case design points. This phenomenon can be explained as follows: Although convexity brings more structure than unimodality, for some worst-case design points this extra structure is uninformative and the nonparametric rates of unimodal regression and convex regression are both $n^{-2/3}$.

Recommendations

• On risk bounds in isotonic and other shape restricted regression problems
• Isotonic regression in general dimensions
• Global risk bounds and adaptation in univariate convex regression
• Nonparametric shape-restricted regression
• The limiting behavior of isotonic and convex regression estimators when the model is misspecified

Cites work

• A new perspective on least squares under convex constraint
• Adaptive risk bounds in unimodal regression
• An improved global risk bound in concave regression
• Computational and statistical tradeoffs via convex relaxation
• Concentration inequalities. A nonasymptotic theory of independence
• Empirical minimization
• Entropy estimate for high-dimensional monotonic functions
• Estimation in high dimensions: a geometric perspective
• Global risk bounds and adaptation in univariate convex regression
• High-dimensional estimation with geometric constraints
• scientific article; zbMATH DE number 49190 (Why is no real title available?)
• Introduction to nonparametric estimation
• Living on the edge: phase transitions in convex programs with random data
• On concentration for (regularized) empirical risk minimization
• On matrix estimation under monotonicity constraints
• On risk bounds in isotonic and other shape restricted regression problems
• On the degrees of freedom in shape-restricted regression.
• Optimal rates of statistical seriation
• Risk bounds in isotonic regression
• Sharp MSE bounds for proximal denoising
• Sharp oracle bounds for monotone and convex regression through aggregation
• Sharp oracle inequalities for least squares estimators in shape restricted regression
• Sharp Time–Data Tradeoffs for Linear Inverse Problems
• Slope meets Lasso: improved oracle bounds and optimality
• The convex geometry of linear inverse problems

Cited in

(52)

• Adaptive risk bounds in unimodal regression
• Optimal rates of statistical seriation
• Nonparametric shape-restricted regression
• Methods for estimation of convex sets
• Sharp oracle inequalities for least squares estimators in shape restricted regression
• Local continuity of log-concave projection, with applications to estimation under model misspecification
• Limit distribution theory for block estimators in multiple isotonic regression
• Isotonic regression in multi-dimensional spaces and graphs
• Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
• Estimating a density, a hazard rate, and a transition intensity via the \(\rho\)-estimation method
• Confidence intervals for multiple isotonic regression and other monotone models
• Suboptimality of constrained least squares and improvements via non-linear predictors
• On least squares estimation under heteroscedastic and heavy-tailed errors
• Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
• Adaptive risk bounds in univariate total variation denoising and trend filtering
• Estimating piecewise monotone signals
• The limiting behavior of isotonic and convex regression estimators when the model is misspecified
• Confidence regions and minimax rates in outlier-robust estimation on the probability simplex
• On estimation of isotonic piecewise constant signals
• Adaptive confidence sets in shape restricted regression
• Discrete minimax estimation with trees
• Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
• Distribution-free properties of isotonic regression
• Isotonic regression in general dimensions
• On the risk of convex-constrained least squares estimators under misspecification
• On risk bounds in isotonic and other shape restricted regression problems
• Adaptation in multivariate log-concave density estimation
• Posterior contraction and testing for multivariate isotonic regression
• Rho-estimators for shape restricted density estimation
• A new perspective on least squares under convex constraint
• scientific article; zbMATH DE number 7626735 (Why is no real title available?)
• Sequential construction and dimension reduction of Gaussian processes under inequality constraints
• scientific article; zbMATH DE number 7306865 (Why is no real title available?)
• Adaptive rates for total variation image denoising
• Oracle inequalities for square root analysis estimators with application to total variation penalties
• Sharp oracle bounds for monotone and convex regression through aggregation
• Coverage of credible intervals in Bayesian multivariate isotonic regression
• Robust oracle estimation and uncertainty quantification for possibly sparse quantiles
• Noisy linear inverse problems under convex constraints: exact risk asymptotics in high dimensions
• Inference for Local Parameters in Convexity Constrained Models
• Minimax analysis for inverse risk in nonparametric planer invertible regression
• Likelihood asymptotics in nonregular settings: a review with emphasis on the likelihood ratio
• A new computational framework for log-concave density estimation
• Isotonic regression meets Lasso
• Statistical and Computational Efficiency for Smooth Tensor Estimation with Unknown Permutations
• Isotonic subgroup selection
• Error bounds for a kernel-based constrained optimal smoothing approximation
• Piecewise monotone estimation in one-parameter exponential families
• Convex regression in multidimensions: suboptimality of least squares estimators
• Empirical Orlicz norms
• High-probability minimax lower bounds
• Regression analysis of stochastic fatigue crack growth model in a martingale difference framework

This page was built for publication: Sharp oracle inequalities for least squares estimators in shape restricted regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1750286)