An analysis of penalized interaction models

DOI10.3150/15-BEJ715MaRDI QIDQ282572zbMATH OpenFDO

Authors Chenlei Leng, Junlong Zhao

Publication date 12 May 2016

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1603.09138, https://projecteuclid.org/euclid.bj/1458133003

zbMATH Keywords

lassohigh-dimensionalityconvergence ratehierarchical variable selectioninteraction modelsrestricted eigenvalue condition

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: An important consideration for variable selection in interaction models is to design an appropriate penalty that respects hierarchy of the importance of the variables. A common theme is to include an interaction term only after the corresponding main effects are present. In this paper, we study several recently proposed approaches and present a unified analysis on the convergence rate for a class of estimators, when the design satisfies the restricted eigenvalue condition. In particular, we show that with probability tending to one, the resulting estimates have a rate of convergence $ssqrtlog{p}_1/n$ in the $el{l}_1$ error, where $p_1$ is the ambient dimension, $s$ is the true dimension and $n$ is the sample size. We give a new proof that the restricted eigenvalue condition holds with high probability, when the variables in the main effects and the errors follow sub-Gaussian distributions. Under this setup, the interactions no longer follow Gaussian or sub-Gaussian distributions even if the main effects follow Gaussian, and thus existing works are not applicable. This result is of independent interest.

Recommendations

• Group regularized estimation under structural hierarchy
• A lasso for hierarchical interactions
• Variable selection with the strong heredity constraint and its oracle property
• Variable selection using adaptive nonlinear interaction structures in high dimensions
• Penalized interaction estimation for ultrahigh dimensional quadratic regression

Cites work

• scientific article; zbMATH DE number 5957408 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• A lasso for hierarchical interactions
• A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
• Component selection and smoothing in multivariate nonparametric regression
• Forward regression for ultra-high dimensional variable screening
• Inequalities for the trace of matrix product
• Interaction screening for ultrahigh-dimensional data
• Lasso-type recovery of sparse representations for high-dimensional data
• Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
• Model Selection and Estimation in Regression with Grouped Variables
• On selecting interacting features from high-dimensional data
• On the conditions used to prove oracle results for the Lasso
• Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
• Reconstruction From Anisotropic Random Measurements
• Restricted eigenvalue properties for correlated Gaussian designs
• Simultaneous analysis of Lasso and Dantzig selector
• Structured sparsity through convex optimization
• Structured variable selection and estimation
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• The commutation matrix: Some properties and applications
• The composite absolute penalties family for grouped and hierarchical variable selection
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• Uniform uncertainty principle for Bernoulli and subgaussian ensembles
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Variable selection using adaptive nonlinear interaction structures in high dimensions
• Variable selection with the strong heredity constraint and its oracle property

Cited in

(8)

• Variable selection and structure identification for varying coefficient Cox models
• HiQR: an efficient algorithm for high-dimensional quadratic regression with penalties
• Group regularized estimation under structural hierarchy
• A lasso for hierarchical interactions
• Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
• High-Dimensional Interaction Detection With False Sign Rate Control
• Variable selection using adaptive nonlinear interaction structures in high dimensions
• Penalized interaction estimation for ultrahigh dimensional quadratic regression

Describes a project that uses

Uses Software

• hierNet

This page was built for publication: An analysis of penalized interaction models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q282572)