On Negative Transfer and Structure of Latent Functions in Multioutput Gaussian Processes

DOI10.1137/21M1436816MaRDI QIDQ6109146zbMATH OpenOpenAlexFDO

Authors

Publication date 30 June 2023

Published in SIAM/ASA Journal on Uncertainty Quantification (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2004.02382

zbMATH Keywords

convolution processmultioutput Gaussian processnegative transfer

Mathematics Subject Classification ID

Gaussian processes (60G15) Inference from stochastic processes and prediction (62M20)

Abstract: The multi-output Gaussian process ($mathcalMGP$) is based on the assumption that outputs share commonalities, however, if this assumption does not hold negative transfer will lead to decreased performance relative to learning outputs independently or in subsets. In this article, we first define negative transfer in the context of an $mathcalMGP$ and then derive necessary conditions for an $mathcalMGP$ model to avoid negative transfer. Specifically, under the convolution construction, we show that avoiding negative transfer is mainly dependent on having a sufficient number of latent functions $Q$ regardless of the flexibility of the kernel or inference procedure used. However, a slight increase in $Q$ leads to a large increase in the number of parameters to be estimated. To this end, we propose two latent structures that scale to arbitrarily large datasets, can avoid negative transfer and allow any kernel or sparse approximations to be used within. These structures also allow regularization which can provide consistent and automatic selection of related outputs.

Recommendations

• Focused multi-task learning in a Gaussian process framework
• Computationally efficient convolved multiple output Gaussian processes
• Multi-task sparse structure learning with Gaussian copula models
• Variational dependent multi-output Gaussian process dynamical systems
• Multi-view Gaussian processes with posterior consistency

Cites work

• \(\alpha\)-variational inference with statistical guarantees
• A generalized Gaussian process model for computer experiments with binary time series
• An efficient surrogate model for emulation and physics extraction of large eddy simulations
• Computationally efficient convolved multiple output Gaussian processes
• Constructing and fitting models for cokriging and multivariable spatial prediction
• Gaussian processes for machine learning.
• Kernels for vector-valued functions: a review
• On Prediction Properties of Kriging: Uniform Error Bounds and Robustness
• Pairwise Fitting of Mixed Models for the Joint Modeling of Multivariate Longitudinal Profiles
• Prediction for computer experiments with both quantitative and qualitative factors
• Space and space-time modeling using process convolutions
• The elements of statistical learning. Data mining, inference, and prediction
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Variational dependent multi-output Gaussian process dynamical systems
• Variational inference for latent variables and uncertain inputs in Gaussian processes

Cited in

(3)

• Focused multi-task learning in a Gaussian process framework
• Computationally efficient convolved multiple output Gaussian processes
• Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators

This page was built for publication: On Negative Transfer and Structure of Latent Functions in Multioutput Gaussian Processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6109146)