An analysis of penalized interaction models

DOI10.3150/15-BEJ715zbMATH Open1360.62392arXiv1603.09138MaRDI QIDQ282572FDOQ282572

Authors: Chenlei Leng, Junlong Zhao 

Publication date: 12 May 2016

Published in: Bernoulli (Search for Journal in Brave)

Abstract: An important consideration for variable selection in interaction models is to design an appropriate penalty that respects hierarchy of the importance of the variables. A common theme is to include an interaction term only after the corresponding main effects are present. In this paper, we study several recently proposed approaches and present a unified analysis on the convergence rate for a class of estimators, when the design satisfies the restricted eigenvalue condition. In particular, we show that with probability tending to one, the resulting estimates have a rate of convergence $ssqrtlog{p}_1/n$ in the $el{l}_1$ error, where $p_1$ is the ambient dimension, $s$ is the true dimension and $n$ is the sample size. We give a new proof that the restricted eigenvalue condition holds with high probability, when the variables in the main effects and the errors follow sub-Gaussian distributions. Under this setup, the interactions no longer follow Gaussian or sub-Gaussian distributions even if the main effects follow Gaussian, and thus existing works are not applicable. This result is of independent interest.

Full work available at URL: https://arxiv.org/abs/1603.09138

Recommendations

• Group regularized estimation under structural hierarchy
• A lasso for hierarchical interactions
• Variable selection with the strong heredity constraint and its oracle property
• Variable selection using adaptive nonlinear interaction structures in high dimensions
• Penalized interaction estimation for ultrahigh dimensional quadratic regression

zbMATH Keywords

lassohigh-dimensionalityconvergence ratehierarchical variable selectioninteraction modelsrestricted eigenvalue condition

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• A lasso for hierarchical interactions
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• Variable selection with the strong heredity constraint and its oracle property
• Lasso-type recovery of sparse representations for high-dimensional data
• On the conditions used to prove oracle results for the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Restricted eigenvalue properties for correlated Gaussian designs
• Title not available (Why is that?)
• Model Selection and Estimation in Regression with Grouped Variables
• A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
• Structured sparsity through convex optimization
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• The composite absolute penalties family for grouped and hierarchical variable selection
• Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
• Component selection and smoothing in multivariate nonparametric regression
• Reconstruction From Anisotropic Random Measurements
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• The commutation matrix: Some properties and applications
• Uniform uncertainty principle for Bernoulli and subgaussian ensembles
• On selecting interacting features from high-dimensional data
• Forward regression for ultra-high dimensional variable screening
• Inequalities for the trace of matrix product
• Interaction screening for ultrahigh-dimensional data
• Variable selection using adaptive nonlinear interaction structures in high dimensions
• Structured variable selection and estimation

Cited In (8)

• HiQR: an efficient algorithm for high-dimensional quadratic regression with penalties
• High-Dimensional Interaction Detection With False Sign Rate Control
• Penalized interaction estimation for ultrahigh dimensional quadratic regression
• Variable selection using adaptive nonlinear interaction structures in high dimensions
• A lasso for hierarchical interactions
• Variable selection and structure identification for varying coefficient Cox models
• Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
• Group regularized estimation under structural hierarchy

Uses Software

• hierNet

This page was built for publication: An analysis of penalized interaction models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q282572)