Signal extraction approach for sparse multivariate response regression

DOI10.1016/J.JMVA.2016.09.005zbMATH Open1351.62110arXiv1508.01105OpenAlexW2278601099MaRDI QIDQ153109FDOQ153109

Authors: Ruiyan Luo, Xin Qi, Ruiyan Luo, Xin Qi 

Publication date: January 2017

Published in: Journal of Multivariate Analysis (Search for Journal in Brave)

Abstract: In this paper, we consider multivariate response regression models with high dimensional predictor variables. One way to model the correlation among the response variables is through the low rank decomposition of the coefficient matrix, which has been considered by several papers for the high dimensional predictors. However, all these papers focus on the singular value decomposition of the coefficient matrix. Our target is the decomposition of the coefficient matrix which leads to the best lower rank approximation to the regression function, the signal part in the response. Given any rank, this decomposition has nearly the smallest expected prediction error among all approximations to the the coefficient matrix with the same rank. To estimate the decomposition, we formulate a penalized generalized eigenvalue problem to obtain the first matrix in the decomposition and then obtain the second one by a least squares method. In the high-dimensional setting, we establish the oracle inequalities for the estimates. Compared to the existing theoretical results, we have less restrictions on the distribution of the noise vector in each observation and allow correlations among its coordinates. Our theoretical results do not depend on the dimension of the multivariate response. Therefore, the dimension is arbitrary and can be larger than the sample size and the dimension of the predictor. Simulation studies and application to real data show that the proposed method has good prediction performance and is efficient in dimension reduction for various reduced rank models.

Full work available at URL: https://arxiv.org/abs/1508.01105

Recommendations

• Multivariate response regression with low-rank and generalized sparsity
• The sparse signomial classification and regression model
• Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection
• Variable selection in high-dimensional sparse multiresponse linear regression models
• Sparse Single Index Models for Multivariate Responses
• Sparse reduced-rank regression for multivariate varying-coefficient models
• Sparse linear identifiable multivariate modeling
• Sparse wavelet regression with multiple predictive curves
• Sparse envelope model: efficient estimation and response variable selection in multivariate linear regression
• Sparse principal component regression via singular value decomposition approach

zbMATH Keywords

dimension reductionmultivariate regressionoracle inequalitiesbest lower rank approximationhigh dimensional predictorssignal extraction

Mathematics Subject Classification ID

Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Simultaneous analysis of Lasso and Dantzig selector
• Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer
• Regularized estimation of large covariance matrices
• Reduced-rank regression for the multivariate linear model
• Multivariate reduced-rank regression
• Sparse reduced-rank regression for simultaneous dimension reduction and variable selection
• Probability in Banach spaces. Isoperimetry and processes
• Optimal selection of reduced rank estimators of high-dimensional matrices
• Reduced Rank Stochastic Regression with a Sparse Singular value Decomposition
• Joint variable and rank selection for parsimonious estimation of high-dimensional matrices
• Input selection and shrinkage in multiresponse linear regression
• Sparse principal component analysis by choice of norm
• Specification and misspecification in reduced rank regression
• Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection

Cited In (5)

• SPReM: sparse projection regression model for high-dimensional linear regression
• Multiple output regression with latent noise
• Sparse Single Index Models for Multivariate Responses
• SiER
• Sparse Reduced Rank Huber Regression in High Dimensions

Uses Software

• UCI-ml

This page was built for publication: Signal extraction approach for sparse multivariate response regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q153109)