A rigorous theory of conditional mean embeddings

DOI10.1137/19M1305069zbMATH Open1493.46043arXiv1912.00671OpenAlexW2991089358MaRDI QIDQ5037567FDOQ5037567

Authors:

Publication date: 1 March 2022

Published in: SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Abstract: Conditional mean embeddings (CMEs) have proven themselves to be a powerful tool in many machine learning applications. They allow the efficient conditioning of probability distributions within the corresponding reproducing kernel Hilbert spaces (RKHSs) by providing a linear-algebraic relation for the kernel mean embeddings of the respective joint and conditional probability distributions. Both centred and uncentred covariance operators have been used to define CMEs in the existing literature. In this paper, we develop a mathematically rigorous theory for both variants, discuss the merits and problems of each, and significantly weaken the conditions for applicability of CMEs. In the course of this, we demonstrate a beautiful connection to Gaussian conditioning in Hilbert spaces.

Full work available at URL: https://arxiv.org/abs/1912.00671

Recommendations

• Kernel Mean Embedding of Distributions: A Review and Beyond
• The linear conditional expectation in Hilbert space
• Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions
• Hilbert space embeddings and metrics on probability measures
• Statistical inference with reproducing kernels

zbMATH Keywords

reproducing kernel Hilbert spaceGaussian measurekernel mean embeddingconditional mean embedding

Mathematics Subject Classification ID

General nonlinear regression (62J02) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22) Set functions and measures and integrals in infinite-dimensional spaces (Wiener measure, Gaussian measure, etc.) (28C20)

Cites Work

• Title not available (Why is that?)
• Hilbert space embeddings and metrics on probability measures
• Title not available (Why is that?)
• Support Vector Machines
• A Hilbert Space Embedding for Distributions
• Universality, Characteristic Kernels and RKHS Embedding of Measures
• Title not available (Why is that?)
• Kernel dimension reduction in regression
• Title not available (Why is that?)
• On operator ranges
• Title not available (Why is that?)
• On Majorization, Factorization, and Range Inclusion of Operators on Hilbert Space
• Title not available (Why is that?)
• Title not available (Why is that?)
• Joint Measures and Cross-Covariance Operators
• An Explicit Description of the Reproducing Kernel Hilbert Spaces of Gaussian RBF Kernels
• Kernel Bayes' rule: Bayesian inference with positive definite kernels
• Theory of reproducing kernels and applications
• Generalized inverses and Douglas equations
• Separability of reproducing kernel spaces
• Title not available (Why is that?)
• Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces

Cited In (7)

• Propagating uncertainty through system dynamics in reproducing kernel Hilbert space
• The linear conditional expectation in Hilbert space
• Critical imbeddings with multivariate rearrangements
• Conditional mean embedding and optimal feature selection via positive definite kernels
• Conditional expectation using compactification operators
• Convergence Rates for Learning Linear Operators from Noisy Data
• Adaptive joint distribution learning

This page was built for publication: A rigorous theory of conditional mean embeddings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5037567)