Sharp oracle inequalities for square root regularization
From MaRDI portal
Abstract: We study a set of regularization methods for high-dimensional linear regression models. These penalized estimators have the square root of the residual sum of squared errors as loss function, and any weakly decomposable norm as penalty function. This fit measure is chosen because of its property that the estimator does not depend on the unknown standard deviation of the noise. On the other hand, a generalized weakly decomposable norm penalty is very useful in being able to deal with different underlying sparsity structures. We can choose a different sparsity inducing norm depending on how we want to interpret the unknown parameter vector . Structured sparsity norms, as defined in Micchelli et al. [18], are special cases of weakly decomposable norms, therefore we also include the square root LASSO (Belloni et al. [3]), the group square root LASSO (Bunea et al. [10]) and a new method called the square root SLOPE (in a similar fashion to the SLOPE from Bogdan et al. [6]). For this collection of estimators our results provide sharp oracle inequalities with the Karush-Kuhn-Tucker conditions. We discuss some examples of estimators. Based on a simulation we illustrate some advantages of the square root SLOPE.
Recommendations
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- Improved bounds for square-root Lasso and square-root slope
- Weakly decomposable regularization penalties and structured sparsity
- Slope meets Lasso: improved oracle bounds and optimality
- Sparsity oracle inequalities for the Lasso
Cites work
- Adaptive estimation of a quadratic functional by model selection.
- Aggregation and Sparsity Via ℓ1 Penalized Least Squares
- Aggregation for Gaussian regression
- Concentration inequalities. A nonasymptotic theory of independence
- Convex analysis and monotone operator theory in Hilbert spaces
- Estimation and testing under sparsity. École d'Été de Probabilités de Saint-Flour XLV -- 2015
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 5251637 (Why is no real title available?)
- Noisy low-rank matrix completion with general sampling distribution
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- Optimization with sparsity-inducing penalties
- Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
- Pivotal estimation via square-root lasso in nonparametric regression
- Regularizers for structured sparsity
- Scaled sparse linear regression
- Simultaneous analysis of Lasso and Dantzig selector
- SLOPE is adaptive to unknown sparsity and asymptotically minimax
- SLOPE-adaptive variable selection via convex optimization
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- Statistics for high-dimensional data. Methods, theory and applications.
- Structured sparsity and generalization
- The Group Square-Root Lasso: Theoretical Properties and Fast Algorithms
- The Lasso, correlated design, and improved oracle inequalities
- Weakly decomposable regularization penalties and structured sparsity
Cited in
(17)- A two-stage regularization method for variable selection and forecasting in high-order interaction model
- Improved bounds for square-root Lasso and square-root slope
- Pivotal estimation via square-root lasso in nonparametric regression
- Correcting for unknown errors in sparse high-dimensional function approximation
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- Oracle inequalities for local and global empirical risk minimizers
- scientific article; zbMATH DE number 7306909 (Why is no real title available?)
- Adaptive rates for total variation image denoising
- Oracle inequalities for square root analysis estimators with application to total variation penalties
- Group sparse recovery via group square-root elastic net and the iterative multivariate thresholding-based algorithm
- A dual semismooth Newton based augmented Lagrangian method for large-scale linearly constrained sparse group square-root Lasso problems
- Square root LASSO: well-posedness, Lipschitz stability, and the tuning trade-off
- Robust and tuning-free sparse linear regression via square-root slope
- Wasserstein distributionally robust optimization and its tractable regularization formulation
- An iterative algorithm for the square-root Lasso
- Outlier robust and sparse estimation of linear regression coefficients
- The distributionally robust prediction error of the \(\sqrt{\mathrm{LASSO}}\) and related estimators
This page was built for publication: Sharp oracle inequalities for square root regularization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4636972)
