Concentration behavior of the penalized least squares estimator

DOI10.1111/STAN.12123MaRDI QIDQ6089165zbMATH OpenOpenAlexFDO

Authors Alan Muro, Sara Van De Geer

Publication date 14 December 2023

Published in Statistica Neerlandica (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1511.08698

zbMATH Keywords

concentration inequalitiesregularized least squaresstatistical trade-off

Mathematics Subject Classification ID

Parametric inference (62Fxx) Nonparametric inference (62Gxx) Linear inference, regression (62Jxx)

Abstract: Consider the standard nonparametric regression model and take as estimator the penalized least squares function. In this article, we study the trade-off between closeness to the true function and complexity penalization of the estimator, where complexity is described by a seminorm on a class of functions. First, we present an exponential concentration inequality revealing the concentration behavior of the trade-off of the penalized least squares estimator around a nonrandom quantity, where such quantity depends on the problem under consideration. Then, under some conditions and for the proper choice of the tuning parameter, we obtain bounds for this nonrandom quantity. We illustrate our results with some examples that include the smoothing splines estimator.

Recommendations

• Towards the study of least squares estimators with convex penalty
• scientific article; zbMATH DE number 1833941
• The Penalized Analytic Center Estimator
• Bounds on the prediction error of penalized least squares estimators with convex penalty
• Penalized likelihood estimation: Convergence under incorrect model
• Consistencies and rates of convergence of jump-penalized least squares estimators
• Penalized least squares estimation with weakly dependent data
• Asymptotic properties of penalized spline estimators in concave extended linear models: rates of convergence

Cites work

• A new perspective on least squares under convex constraint
• Bounds on the prediction error of penalized least squares estimators with convex penalty
• Concentration inequalities. A nonasymptotic theory of independence
• Empirical minimization
• Estimating a regression function
• scientific article; zbMATH DE number 428352 (Why is no real title available?)
• scientific article; zbMATH DE number 45848 (Why is no real title available?)
• scientific article; zbMATH DE number 1420699 (Why is no real title available?)
• Information-theoretic determination of minimax rates of convergence
• Lectures on empirical processes. Theory and statistical applications.
• Maxiset in sup-norm for kernel estimators
• Minimax or maxisets?
• On concentration for (regularized) empirical risk minimization
• Optimal global rates of convergence for nonparametric regression
• Optimal model selection in heteroscedastic regression using piecewise polynomial functions
• Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression
• Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
• Slope heuristics and V-fold model selection in heteroscedastic regression using strongly localized bases
• Smoothing spline ANOVA models
• Weak convergence and empirical processes. With applications to statistics

Cited in

(1)

• The Penalized Analytic Center Estimator

This page was built for publication: Concentration behavior of the penalized least squares estimator

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6089165)