Oracle inequalities and optimal inference under group sparsity

DOI10.1214/11-AOS896zbMATH Open1306.62156arXiv1007.1771OpenAlexW3122814465MaRDI QIDQ651028FDOQ651028

Authors: K. Lounici, Massimiliano Pontil, Sara Van De Geer, Alexandre B. Tsybakov 

Publication date: 8 December 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We consider the problem of estimating a sparse linear regression vector $$ under a gaussian noise model, for the purpose of both prediction and model selection. We assume that prior knowledge is available on the sparsity pattern, namely the set of variables is partitioned into prescribed groups, only few of which are relevant in the estimation process. This group sparsity assumption suggests us to consider the Group Lasso method as a means to estimate $$. We establish oracle inequalities for the prediction and $el{l}_2$ estimation errors of this estimator. These bounds hold under a restricted eigenvalue condition on the design matrix. Under a stronger coherence condition, we derive bounds for the estimation error for mixed $(2,p)$-norms with $1lepleqinfty$. When $p=infty$, this result implies that a threshold version of the Group Lasso estimator selects the sparsity pattern of $$ with high probability. Next, we prove that the rate of convergence of our upper bounds is optimal in a minimax sense, up to a logarithmic factor, for all estimators over a class of group sparse vectors. Furthermore, we establish lower bounds for the prediction and $el{l}_2$ estimation errors of the usual Lasso estimator. Using this result, we demonstrate that the Group Lasso can achieve an improvement in the prediction and estimation properties as compared to the Lasso.

Full work available at URL: https://arxiv.org/abs/1007.1771

Recommendations

• Some theoretical results on the grouped variables Lasso
• On the asymptotic properties of the group lasso estimator for linear models
• Non-asymptotic oracle inequalities for the Lasso and group Lasso in high dimensional logistic model
• Sparsity oracle inequalities for the Lasso
• The benefit of group sparsity

zbMATH Keywords

statistical learningminimax riskoracle inequalitiespenalized least squaresmoment inequalitygroup sparsitygroup lasso

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Statistical ranking and selection procedures (62F07) Minimax procedures in statistical decision theory (62C20)

Cites Work

• High-dimensional additive modeling
• Weak convergence and empirical processes. With applications to statistics
• Simultaneous analysis of Lasso and Dantzig selector
• High-dimensional generalized linear models and the lasso
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Econometric analysis of cross section and panel data.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• Model Selection and Estimation in Regression with Grouped Variables
• An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for Aggregation Bias
• Sparsity oracle inequalities for the Lasso
• Nemirovski's inequalities revisited
• The Group Lasso for Logistic Regression
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• The composite absolute penalties family for grouped and hierarchical variable selection
• Title not available (Why is that?)
• Stable recovery of sparse overcomplete representations in the presence of noise
• Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
• Variable selection in nonparametric additive models
• Aggregation for Gaussian regression
• Consistency of the group Lasso and multiple kernel learning
• Title not available (Why is that?)
• The benefit of group sparsity
• Oracle inequalities for inverse problems
• On the asymptotic properties of the group lasso estimator for linear models
• Exponential screening and optimal rates of sparse estimation
• Some theoretical results on the grouped variables Lasso
• Moment inequalities for sums of dependent random variables under projective conditions
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Title not available (Why is that?)
• Introduction to nonparametric estimation
• Convex analysis and nonlinear optimization. Theory and examples.
• Support union recovery in high-dimensional multivariate regression
• Bounds for linear multi-task learning

Cited In (only showing first 100 items - show all)

• ``Grouping strategies and thresholding for high dimensional linear models: discussion
• ``Grouping strategies and thresholding for high dimensional linear models: rejoinder
• Robust regression via mutivariate regression depth
• Grouping strategies and thresholding for high dimensional linear models
• Solution of linear ill-posed problems using overcomplete dictionaries
• Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
• Least squares after model selection in high-dimensional sparse models
• Title not available (Why is that?)
• Non-asymptotic oracle inequalities for the Lasso and group Lasso in high dimensional logistic model
• Some theoretical results on the grouped variables Lasso
• Statistical and computational limits for sparse matrix detection
• Structured variable selection via prior-induced hierarchical penalty functions
• Oracle inequalities for weighted group Lasso in high-dimensional misspecified Cox models
• Tuning-free heterogeneous inference in massive networks
• Title not available (Why is that?)
• A no-free-lunch theorem for multitask learning
• Greedy variance estimation for the LASSO
• Randomized maximum-contrast selection: subagging for large-scale regression
• Slope meets Lasso: improved oracle bounds and optimality
• Title not available (Why is that?)
• Generalized linear models with structured sparsity estimators
• Model selection in regression under structural constraints
• Multivariate sparse group Lasso for the multivariate multiple linear regression with an arbitrary group structure
• Theoretical properties of the overlapping groups Lasso
• Estimation and variable selection with exponential weights
• Sharp oracle inequalities for low-complexity priors
• Gelfand numbers related to structured sparsity and Besov space embeddings with small mixed smoothness
• Prediction and estimation consistency of sparse multi-class penalized optimal scoring
• High-dimensional generalized linear models incorporating graphical structure among predictors
• Bayesian MIDAS penalized regressions: estimation, selection, and prediction
• Optimal estimation and rank detection for sparse spiked covariance matrices
• Sparse PCA: optimal rates and adaptive estimation
• Bayesian linear regression for multivariate responses under group sparsity
• Variable selection in heterogeneous panel data models with cross‐sectional dependence
• Variable selection, monotone likelihood ratio and group sparsity
• Simultaneous feature selection and clustering based on square root optimization
• Tight conditions for consistency of variable selection in the context of high dimensionality
• A general framework for Bayes structured linear models
• Trace regression model with simultaneously low rank and row(column) sparse parameter
• Regularizers for structured sparsity
• Sparse regression and support recovery with \(\mathbb{L}_2\)-boosting algorithms
• Structured estimation for the nonparametric Cox model
• Sparse high-dimensional varying coefficient model: nonasymptotic minimax study
• Robust grouped variable selection using distributionally robust optimization
• A varying-coefficient panel data model with fixed effects: theory and an application to US commercial banks
• Lasso in Infinite dimension: application to variable selection in functional multivariate linear regression
• Variable selection and structure identification for varying coefficient Cox models
• Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean
• The benefit of group sparsity in group inference with de-biased scaled group Lasso
• A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
• Pivotal estimation via square-root lasso in nonparametric regression
• Accuracy guaranties for \(\ell_{1}\) recovery of block-sparse signals
• Robust inference on average treatment effects with possibly more covariates than observations
• Log-Contrast Regression with Functional Compositional Predictors: Linking Preterm Infant's Gut Microbiome Trajectories to Neurobehavioral Outcome
• A note on coding and standardization of categorical variables in (sparse) group Lasso regression
• Monte Carlo simulation for Lasso-type problems by estimator augmentation
• Structured sparsity through convex optimization
• Support union recovery in high-dimensional multivariate regression
• A group adaptive elastic-net approach for variable selection in high-dimensional linear regression
• Estimation of high-dimensional low-rank matrices
• Joint variable and rank selection for parsimonious estimation of high-dimensional matrices
• Group regularized estimation under structural hierarchy
• A selective review of group selection in high-dimensional models
• Sparse estimation by exponential weighting
• Decomposable norm minimization with proximal-gradient homotopy algorithm
• The de-biased group Lasso estimation for varying coefficient models
• Robust regression with compositional covariates
• Consistent group selection with Bayesian high dimensional modeling
• High-dimensional regression with unknown variance
• High-Dimensional Gaussian Graphical Regression Models with Covariates
• Bayesian Multi-Task Variable Selection with an Application to Differential DAG Analysis
• Group sparse recovery via group square-root elastic net and the iterative multivariate thresholding-based algorithm
• Noise covariance estimation in multi-task high-dimensional linear models
• Low-rank matrix recovery under heavy-tailed errors
• An Interactive Greedy Approach to Group Sparsity in High Dimensions
• ISLET: fast and optimal low-rank tensor regression via importance sketching
• Group sparse structural smoothing recovery: model, statistical properties and algorithm
• Title not available (Why is that?)
• Multi-Task Learning with High-Dimensional Noisy Images
• Variable selection for generalized linear model with highly correlated covariates
• Joint association and classification analysis of multi‐view data
• Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach
• Discrete optimization methods for group model selection in compressed sensing
• Posterior contraction in group sparse logit models for categorical responses
• Adaptive estimation in multivariate response regression with hidden variables
• Oracle inequalities for weighted group Lasso in high-dimensional Poisson regression model
• Grouped variable selection with discrete optimization: computational and statistical perspectives
• Adaptive estimation of the rank of the coefficient matrix in high-dimensional multivariate response regression models
• Minimax estimation in multi-task regression under low-rank structures
• Adaptive and robust multi-task learning
• Penalized least square in sparse setting with convex penalty and non Gaussian errors
• On the finite-sample analysis of \(\Theta\)-estimators
• Expectile trace regression via low-rank and group sparsity regularization
• Simultaneous off-the-grid learning of mixtures issued from a continuous dictionary
• Improved linear regression prediction by transfer learning
• A Unified Approach to Sparse Tweedie Modeling of Multisource Insurance Claim Data
• The statistical rate for support matrix machines under low rankness and row (column) sparsity
• Adaptive bi-level variable selection for multivariate failure time model with a diverging number of covariates
• A penalized two-pass regression to predict stock returns with time-varying risk premia
• On the finite-sample analysis of \(\Theta\)-estimators

Uses Software

• hgam

This page was built for publication: Oracle inequalities and optimal inference under group sparsity

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q651028)