Multi-target regression via input space expansion: treating targets as inputs

DOI10.1007/S10994-016-5546-ZzbMATH Open1454.68134DBLPjournals/ml/XioufisTGV16arXiv1211.6581OpenAlexW3124484778WikidataQ62864526 ScholiaQ62864526MaRDI QIDQ1689552FDOQ1689552

Ioannis Vlahavas, Eleftherios Spyromitros-Xioufis, William Groves, Grigorios Tsoumakas

Publication date: 12 January 2018

Published in: Machine Learning (Search for Journal in Brave)

Abstract: In many practical applications of supervised learning the task involves the prediction of multiple target variables from a common set of input variables. When the prediction targets are binary the task is called multi-label classification, while when the targets are continuous the task is called multi-target regression. In both tasks, target variables often exhibit statistical dependencies and exploiting them in order to improve predictive accuracy is a core challenge. A family of multi-label classification methods address this challenge by building a separate model for each target on an expanded input space where other targets are treated as additional input variables. Despite the success of these methods in the multi-label classification domain, their applicability and effectiveness in multi-target regression has not been studied until now. In this paper, we introduce two new methods for multi-target regression, called Stacked Single-Target and Ensemble of Regressor Chains, by adapting two popular multi-label classification methods of this family. Furthermore, we highlight an inherent problem of these methods - a discrepancy of the values of the additional input variables between training and prediction - and develop extensions that use out-of-sample estimates of the target variables during training in order to tackle this problem. The results of an extensive experimental evaluation carried out on a large and diverse collection of datasets show that, when the discrepancy is appropriately mitigated, the proposed methods attain consistent improvements over the independent regressions baseline. Moreover, two versions of Ensemble of Regression Chains perform significantly better than four state-of-the-art methods including regularization-based multi-task learning methods and a multi-objective random forest approach.

Full work available at URL: https://arxiv.org/abs/1211.6581

Recommendations

• Multi-target prediction: a unifying view on problems and methods
• scientific article; zbMATH DE number 6276187
• Multi-target support vector regression via correlation regressor chains
• Multi-task regression using minimal penalties
• Ensembles for multi-target regression with random output selections
• Multi-output regression on the output manifold
• Feature ranking for multi-target regression
• Least square regularized regression for multitask learning
• Multi-label classification via multi-target regression on data streams

zbMATH Keywords

multi-label classificationchainingstackingmulti-target regression

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• First order regression
• The elements of statistical learning. Data mining, inference, and prediction
• Title not available (Why is that?)
• Bagging predictors
• Title not available (Why is that?)
• Title not available (Why is that?)
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Kernels for vector-valued functions: a review
• Modern Multivariate Statistical Techniques
• Title not available (Why is that?)
• Reduced-rank regression for the multivariate linear model
• Stochastic gradient boosting.
• Title not available (Why is that?)
• Approximations of the critical region of the fbietkan statistic
• Not all traces on the circle come from functions of least gradient in the disk
• Convex multi-task feature learning
• Combining instance-based learning and logistic regression for multilabel classification
• On label dependence and loss minimization in multi-label classification
• Title not available (Why is that?)
• Efficient Monte Carlo methods for multi-dimensional learning with classifier chains
• Multivariate regression analysis and canonical variates
• Multilabel Classification with Principal Label Space Transformation
• Title not available (Why is that?)
• Multi-target regression via input space expansion: treating targets as inputs
• A Dirty Model for Multiple Sparse Regression

Cited In (25)

• Prediction of arch dam deformation via correlated multi-target stacking
• Fast and accurate pseudoinverse with sparse matrix reordering and incremental approach
• Incremental predictive clustering trees for online semi-supervised multi-target regression
• Scalable Model-Free Feature Screening via Sliced-Wasserstein Dependency
• Regularization-based model tree for multi-output regression
• A deep multitask learning approach for air quality prediction
• Progressive random \(k\)-labelsets for cost-sensitive multi-label classification
• Multi-label classification with weighted classifier selection and stacked ensemble
• Title not available (Why is that?)
• Learning-augmented heuristics for scheduling parallel serial-batch processing machines
• Multi-target prediction: a unifying view on problems and methods
• Model averaging for sparse seemingly unrelated regression using Bayesian networks among the errors
• An ensemble-adaptive tree-based chain framework for multi-target regression problems
• Alternating DCA for reduced-rank multitask linear regression with covariance matrix estimation
• Orthogonal canonical correlation analysis and applications
• Multi-target support vector regression via correlation regressor chains
• Ensembles for multi-target regression with random output selections
• A modified primal-dual algorithm for matrix completion problems
• Robust multi-target regression with improved stochastic configuration networks and its applications
• Feature ranking for multi-target regression
• Multi-label classification via multi-target regression on data streams
• An improved inertial alternating direction method for low rank matrix completion problems
• Multi-target regression via input space expansion: treating targets as inputs
• Parsimonious mixture-of-experts based on mean mixture of multivariate normal distributions
• JGPR: a computationally efficient multi-target Gaussian process regression algorithm

Uses Software

• UCI-ml
• FORS
• MULAN
• MALSAR

This page was built for publication: Multi-target regression via input space expansion: treating targets as inputs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1689552)