Kernel Mean Embedding of Distributions: A Review and Beyond

DOI10.1561/2200000060zbMATH Open1380.68336DBLPjournals/ftml/MuandetFSS17arXiv1605.09522OpenAlexW2418306335WikidataQ57340873 ScholiaQ57340873MaRDI QIDQ4593744FDOQ4593744

Authors: Krikamol Muandet, Kenji Fukumizu, Bernhard Schölkopf, Bharath K. Sriperumbudur 

Publication date: 22 November 2017

Published in: Foundations and Trends® in Machine Learning (Search for Journal in Brave)

Abstract: A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be extended to probability measures. It can be viewed as a generalization of the original "feature map" common to support vector machines (SVMs) and other kernel methods. While initially closely associated with the latter, it has meanwhile found application in fields ranging from kernel machines and probabilistic modeling to statistical inference, causal discovery, and deep learning. The goal of this survey is to give a comprehensive review of existing work and recent advances in this research area, and to discuss the most challenging issues and open problems that could lead to new research directions. The survey begins with a brief introduction to the RKHS and positive definite kernels which forms the backbone of this survey, followed by a thorough discussion of the Hilbert space embedding of marginal distributions, theoretical guarantees, and a review of its applications. The embedding of distributions enables us to apply RKHS methods to probability measures which prompts a wide range of applications such as kernel two-sample testing, independent testing, and learning on distributional data. Next, we discuss the Hilbert space embedding for conditional distributions, give theoretical insights, and review some applications. The conditional mean embedding enables us to perform sum, product, and Bayes' rules---which are ubiquitous in graphical model, probabilistic inference, and reinforcement learning---in a non-parametric way. We then discuss relationships between this framework and other related areas. Lastly, we give some suggestions on future research directions.

Full work available at URL: https://arxiv.org/abs/1605.09522

Recommendations

• Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions
• Minimax estimation of kernel mean embeddings
• Admissible kernels for RKHS embedding of probability distributions
• scientific article
• scientific article; zbMATH DE number 7626782
• scientific article; zbMATH DE number 751035
• Kernel estimators of density in spaces of an arbitrary nature.
• scientific article; zbMATH DE number 1530929
• On kernel estimation of a multivariate distribution function

zbMATH Keywords

identificationstatisticsestimationkernel methodsnonparametric methodspattern recognitionfilteringinformation theorystatistical learning theorystatistical/machine learning

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22)

Cited In (71)

• A rigorous theory of conditional mean embeddings
• Estimation of Copulas via Maximum Mean Discrepancy
• Supervised learning of sheared distributions using linearized optimal transport
• Distribution regression model with a Reproducing Kernel Hilbert Space approach
• Estimation with infinite-dimensional exponential family and Fisher divergence
• Title not available (Why is that?)
• Convergence analysis of deterministic kernel-based quadrature rules in misspecified settings
• Optimal Monte Carlo integration on closed manifolds
• Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces
• Local permutation tests for conditional independence
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Nonlinear directed acyclic graph estimation based on the kernel partial correlation coefficient
• State-based confidence bounds for data-driven stochastic reachability using Hilbert space embeddings
• A preconditioning technique for Krylov subspace methods in RKHSs
• Title not available (Why is that?)
• Learning rates for the kernel regularized regression with a differentiable strongly convex loss
• Hawkes Processes Modeling, Inference, and Control: An Overview
• Hilbert space embeddings and metrics on probability measures
• Training image free high-order stochastic simulation based on aggregated kernel statistics
• Title not available (Why is that?)
• Model-free inference of diffusion networks using RKHS embeddings
• Toward a Kernel-Based Uncertainty Decomposition Framework for Data and Models
• Evaluating Forecasts for High-Impact Events Using Transformed Kernel Scores
• A Hilbert Space Embedding for Distributions
• Comparing a large number of multivariate distributions
• High-order sequential simulation via statistical learning in reproducing kernel Hilbert space
• Reproducing kernel Hilbert spaces in the mean field limit
• Title not available (Why is that?)
• Strictly proper kernel scores and characteristic kernels on compact spaces
• Characteristic and universal tensor product kernels
• Bayesian optimization with approximate set kernels
• Sparse machine learning in Banach spaces
• Kernel methods for detecting coherent structures in dynamical data
• Title not available (Why is that?)
• Optimization hierarchy for fair statistical decision problems
• Kernel method for persistence diagrams via kernel embedding and weight factor
• Topology, convergence, and reconstruction of predictive states
• Admissible kernels for RKHS embedding of probability distributions
• Model-based kernel sum rule: kernel Bayesian inference with probabilistic models
• Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
• Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifolds
• Near-optimal coresets of kernel density estimates
• Gaussian processes with multidimensional distribution inputs via optimal transport and Hilbertian embedding
• A Generalized Kernel Method for Global Sensitivity Analysis
• Linear response based parameter estimation in the presence of model error
• Reproducing kernel Hilbert space embedding for adaptive estimation of nonlinearities in piezoelectric systems
• Krylov subspace methods for estimating operator-vector multiplications in Hilbert spaces
• Nonasymptotic one- and two-sample tests in high dimension with unknown covariance structure
• Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces
• Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions
• Hypothesis Testing for Matched Pairs with Missing Data by Maximum Mean Discrepancy: An Application to Continuous Glucose Monitoring
• On nonparametric conditional independence tests for continuous variables
• Unbalanced optimal transport and maximum mean discrepancies: interconnections and rapid evaluation
• Spectral regularized Kernel two-sample tests
• Nonconvex SVM for cancer diagnosis based on morphologic features of tumor microenvironment
• Coefficient-based regularized distribution regression
• Approximating symmetrized estimators of scatter via balanced incomplete \(U\)-statistics
• Sharp Analysis of Sketch-and-Project Methods via a Connection to Randomized Singular Value Decomposition
• Scalable kernel two-sample tests via empirical likelihood and jackknife
• Kernel-based Sensitivity Analysis for (Excursion) Sets
• Error analysis of kernel/GP methods for nonlinear and parametric PDEs
• Energy-based sequential sampling for low-rank PSD-matrix approximation
• A Gaussian process approach to model checks
• A kernel framework for learning differential equations and their solution operators
• Kernel embedding of measures and low-rank approximation of integral operators
• Kernel mean embedding vs kernel density estimation: a quantum perspective
• Reversible Gromov-Monge sampler for simulation-based inference
• Reinforcement learning in non-Markovian environments
• Closed-form expressions for maximum mean discrepancy with applications to Wasserstein auto-encoders

This page was built for publication: Kernel Mean Embedding of Distributions: A Review and Beyond

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4593744)