Kernel Mean Embedding of Distributions: A Review and Beyond
From MaRDI portal
Abstract: A Hilbert space embedding of a distribution---in short, a kernel mean embedding---has recently emerged as a powerful tool for machine learning and inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be extended to probability measures. It can be viewed as a generalization of the original "feature map" common to support vector machines (SVMs) and other kernel methods. While initially closely associated with the latter, it has meanwhile found application in fields ranging from kernel machines and probabilistic modeling to statistical inference, causal discovery, and deep learning. The goal of this survey is to give a comprehensive review of existing work and recent advances in this research area, and to discuss the most challenging issues and open problems that could lead to new research directions. The survey begins with a brief introduction to the RKHS and positive definite kernels which forms the backbone of this survey, followed by a thorough discussion of the Hilbert space embedding of marginal distributions, theoretical guarantees, and a review of its applications. The embedding of distributions enables us to apply RKHS methods to probability measures which prompts a wide range of applications such as kernel two-sample testing, independent testing, and learning on distributional data. Next, we discuss the Hilbert space embedding for conditional distributions, give theoretical insights, and review some applications. The conditional mean embedding enables us to perform sum, product, and Bayes' rules---which are ubiquitous in graphical model, probabilistic inference, and reinforcement learning---in a non-parametric way. We then discuss relationships between this framework and other related areas. Lastly, we give some suggestions on future research directions.
Recommendations
- Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions
- Minimax estimation of kernel mean embeddings
- Admissible kernels for RKHS embedding of probability distributions
- scientific article; zbMATH DE number 4178397
- scientific article; zbMATH DE number 7626782
- scientific article; zbMATH DE number 751035
- Kernel estimators of density in spaces of an arbitrary nature.
- scientific article; zbMATH DE number 1530929
- On kernel estimation of a multivariate distribution function
Cited in
(only showing first 100 items - show all)- Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifolds
- Bayesian optimization with approximate set kernels
- Admissible kernels for RKHS embedding of probability distributions
- Training image free high-order stochastic simulation based on aggregated kernel statistics
- Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
- Supervised learning of sheared distributions using linearized optimal transport
- Optimization hierarchy for fair statistical decision problems
- Local permutation tests for conditional independence
- State-based confidence bounds for data-driven stochastic reachability using Hilbert space embeddings
- Linear response based parameter estimation in the presence of model error
- A preconditioning technique for Krylov subspace methods in RKHSs
- Near-optimal coresets of kernel density estimates
- Learning rates for the kernel regularized regression with a differentiable strongly convex loss
- Gaussian processes with multidimensional distribution inputs via optimal transport and Hilbertian embedding
- High-order sequential simulation via statistical learning in reproducing kernel Hilbert space
- Comparing a large number of multivariate distributions
- Model-free inference of diffusion networks using RKHS embeddings
- Krylov subspace methods for estimating operator-vector multiplications in Hilbert spaces
- Convergence analysis of deterministic kernel-based quadrature rules in misspecified settings
- Optimal Monte Carlo integration on closed manifolds
- Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces
- Strictly proper kernel scores and characteristic kernels on compact spaces
- Topology, convergence, and reconstruction of predictive states
- Hilbert space embeddings and metrics on probability measures
- A Hilbert Space Embedding for Distributions
- Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions
- Kernel method for persistence diagrams via kernel embedding and weight factor
- Characteristic and universal tensor product kernels
- On the optimality of kernel-embedding based goodness-of-fit tests
- Multi-class Gaussian process classification with noisy inputs
- Toward a Kernel-Based Uncertainty Decomposition Framework for Data and Models
- A rigorous theory of conditional mean embeddings
- scientific article; zbMATH DE number 7626782 (Why is no real title available?)
- Distribution regression model with a Reproducing Kernel Hilbert Space approach
- Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces
- Cramer-Wold auto-encoder
- scientific article; zbMATH DE number 7306872 (Why is no real title available?)
- scientific article; zbMATH DE number 7415079 (Why is no real title available?)
- The ridgelet prior: a covariance function approach to prior specification for Bayesian neural networks
- Counterfactual mean embeddings
- Kernel methods for detecting coherent structures in dynamical data
- A Generalized Kernel Method for Global Sensitivity Analysis
- Hawkes Processes Modeling, Inference, and Control: An Overview
- Evaluating Forecasts for High-Impact Events Using Transformed Kernel Scores
- Estimation of Copulas via Maximum Mean Discrepancy
- scientific article; zbMATH DE number 7758314 (Why is no real title available?)
- Reproducing kernel Hilbert spaces in the mean field limit
- Sparse machine learning in Banach spaces
- Reproducing kernel Hilbert space embedding for adaptive estimation of nonlinearities in piezoelectric systems
- Nonasymptotic one- and two-sample tests in high dimension with unknown covariance structure
- Estimation with infinite-dimensional exponential family and Fisher divergence
- Nonlinear directed acyclic graph estimation based on the kernel partial correlation coefficient
- Coefficient-based regularized distribution regression
- Approximating symmetrized estimators of scatter via balanced incomplete \(U\)-statistics
- Sharp Analysis of Sketch-and-Project Methods via a Connection to Randomized Singular Value Decomposition
- Scalable kernel two-sample tests via empirical likelihood and jackknife
- A kernel framework for learning differential equations and their solution operators
- Kernel embedding of measures and low-rank approximation of integral operators
- Reversible Gromov-Monge sampler for simulation-based inference
- Reinforcement learning in non-Markovian environments
- Closed-form expressions for maximum mean discrepancy with applications to Wasserstein auto-encoders
- Hypothesis Testing for Matched Pairs with Missing Data by Maximum Mean Discrepancy: An Application to Continuous Glucose Monitoring
- On nonparametric conditional independence tests for continuous variables
- Unbalanced optimal transport and maximum mean discrepancies: interconnections and rapid evaluation
- Spectral regularized Kernel two-sample tests
- Nonconvex SVM for cancer diagnosis based on morphologic features of tumor microenvironment
- Kernel-based Sensitivity Analysis for (Excursion) Sets
- Error analysis of kernel/GP methods for nonlinear and parametric PDEs
- Energy-based sequential sampling for low-rank PSD-matrix approximation
- A Gaussian process approach to model checks
- Kernel mean embedding vs kernel density estimation: a quantum perspective
- Doubly robust conditional independence testing with generative neural networks
- Compositional data graphical model learning based on conditional independence measurement
- Signal reconstruction using determinantal sampling
- Minimax optimal goodness-of-fit testing with kernel Stein discrepancy
- Randomization tests for conditional group symmetry
- Finer-grained hardness of kernel density estimation
- On the convergence rate of two-stage sampling distribution regression
- Projection pursuit via kernel mean embeddings
- Wasserstein gradient flows for Moreau envelopes of f-divergences in reproducing kernel Hilbert spaces
- Density estimation using the perceptron
- Learning conditional distributions on continuous spaces
- Efficient numerical integration in reproducing kernel Hilbert spaces via leverage scores sampling
- Distribution free tests for model selection based on maximum mean discrepancy with estimated parameters
- Koopman operators with intrinsic observables in rigged reproducing kernel Hilbert spaces
- Nearest neighbors GParareal: improving scalability of Gaussian processes for parallel-in-time solvers
- Learning causal effect of physical activity distribution: an application of functional treatment effect estimation with unmeasured confounding
- Enhancing high-dimensional probabilistic model updating: a generic generative model-inspired framework with GAN-embedded implementation
- Statistical depth meets machine learning: kernel mean embeddings and depth in functional data analysis
- A kernel-based test for the first-order separability of spatio-temporal point processes
- Improved learning theory for kernel distribution regression with two-stage sampling
- On the probabilistic approximation in reproducing kernel Hilbert spaces
- Diffeomorphic measure matching with kernels for generative modeling
- Inferring kernel -machines: discovering structure in complex systems
- Linear quadratic control of nonlinear systems with Koopman operator learning and the Nyström method
- Samplets: wavelet concepts for scattered data
- An approximation theory framework for measure-transport sampling algorithms
- Minimum kernel discrepancy estimators
- Concentration of discrepancy-based approximate Bayesian computation via Rademacher complexity
- Confidence-based estimators for predictive performance in model monitoring
This page was built for publication: Kernel Mean Embedding of Distributions: A Review and Beyond
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4593744)
