Multi-task regression using minimal penalties

MaRDI QIDQ5405207zbMATH OpenFDO

Authors Matthieu Solnon, Sylvain Arlot, Francis Bach

Publication date 1 April 2014

Full work available at URL https://arxiv.org/abs/1107.4512, http://www.jmlr.org/papers/v13/solnon12a.html

zbMATH Keywords

learning theoryoracle inequalitymulti-task

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Learning and adaptive systems in artificial intelligence (68T05) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: In this paper we study the kernel multiple ridge regression framework, which we refer to as multi-task regression, using penalization techniques. The theoretical analysis of this problem shows that the key element appearing for an optimal calibration is the covariance matrix of the noise between the different tasks. We present a new algorithm to estimate this covariance matrix, based on the concept of minimal penalty, which was previously used in the single-task regression framework to estimate the variance of the noise. We show, in a non-asymptotic setting and under mild assumptions on the target function, that this estimator converges towards the covariance matrix. Then plugging this estimator into the corresponding ideal penalty leads to an oracle inequality. We illustrate the behavior of our algorithm on synthetic examples.

Recommendations

• Low rank multivariate regression
• Multi-task learning via linear functional strategy
• Learning multiple tasks with kernel methods
• Least square regularized regression for multitask learning
• Bounds for linear multi-task learning

Cited in

(8)

• Union support recovery in multi-task learning
• Revisiting Stein's paradox: multi-task averaging
• Error analysis on Hérmite learning with gradient data
• Minimax estimation in multi-task regression under low-rank structures
• Sparse regression and support recovery with \(\mathbb{L}_2\)-boosting algorithms
• Alternating DCA for reduced-rank multitask linear regression with covariance matrix estimation
• Multi-target regression via input space expansion: treating targets as inputs
• Least square regularized regression for multitask learning

This page was built for publication: Multi-task regression using minimal penalties

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5405207)