A cluster elastic net for multivariate regression

DOI10.48550/ARXIV.1707.03530zbMATH Open1473.62243arXiv1707.03530MaRDI QIDQ63195FDOQ63195

Authors: Bradley S. Price, Ben Sherwood, Bradley S. Price, Ben Sherwood 

Publication date: 12 July 2017

Abstract: We propose a method for estimating coefficients in multivariate regression when there is a clustering structure to the response variables. The proposed method includes a fusion penalty, to shrink the difference in fitted values from responses in the same cluster, and an L1 penalty for simultaneous variable selection and estimation. The method can be used when the grouping structure of the response variables is known or unknown. When the clustering structure is unknown the method will simultaneously estimate the clusters of the response and the regression coefficients. Theoretical results are presented for the penalized least squares case, including asymptotic results allowing for p >> n. We extend our method to the setting where the responses are binomial variables. We propose a coordinate descent algorithm for both the normal and binomial likelihood, which can easily be extended to other generalized linear model (GLM) settings. Simulations and data examples from business operations and genomics are presented to show the merits of both the least squares and binomial methods.

Full work available at URL: https://arxiv.org/abs/1707.03530

Recommendations

• MCEN: a method of simultaneous variable selection and clustering for high-dimensional multinomial regression
• scientific article; zbMATH DE number 6458363
• Clusterwise functional linear regression models
• Sparse regression with exact clustering
• Simultaneous multiple response regression and inverse covariance matrix estimation via penalized Gaussian maximum likelihood

zbMATH Keywords

clusteringfusion penaltymultivariate regression

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Linear regression; mixed models (62J05) Learning and adaptive systems in artificial intelligence (68T05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Correlated variables in regression: clustering and sparse estimation
• Title not available (Why is that?)
• Lasso-type recovery of sparse representations for high-dimensional data
• On the conditions used to prove oracle results for the Lasso
• Simultaneous analysis of Lasso and Dantzig selector
• Adaptive piecewise polynomial estimation via trend filtering
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• A new approach to Cholesky-based covariance regularization in high dimensions
• Envelope models for parsimonious and efficient multivariate linear regression
• Title not available (Why is that?)
• Simultaneous multiple response regression and inverse covariance matrix estimation via penalized Gaussian maximum likelihood
• Sparsity and Smoothness Via the Fused Lasso
• Extending the linear model with R. Generalized linear, mixed effects and nonparametric regression models.
• Regularization and Variable Selection Via the Elastic Net
• SPReM: sparse projection regression model for high-dimensional linear regression
• Foundations for envelope models and methods
• Indirect multivariate response linear regression
• Model Selection and Estimation in Regression with Grouped Variables
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• A significance test for graph-constrained estimation
• A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
• Variable selection and regression analysis for graph-structured covariates with an application to genomics
• The sparse Laplacian shrinkage estimator for high-dimensional regression
• Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping
• Properties and refinements of the fused Lasso
• Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer

Cited In (9)

• A new globally convergent algorithm for non-Lipschitz \(\ell_{p}-\ell_q\) minimization
• Sparse regression with exact clustering
• Clusterwise elastic-net regression based on a combined information criterion
• Grouping Variable Selection by Weight Fused Elastic Net for Multi-Collinear Data
• Smooth and locally sparse estimation for multiple-output functional linear regression
• Group linear algorithm with sparse principal decomposition: a variable selection and clustering method for generalized linear models
• mcen
• Asymptotic properties of concave \(L_1\)-norm group penalties
• MCEN: a method of simultaneous variable selection and clustering for high-dimensional multinomial regression

This page was built for publication: A cluster elastic net for multivariate regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q63195)