Minimax estimation of kernel mean embeddings

MaRDI QIDQ4636999zbMATH OpenFDO

Authors Ilya Tolstikhin, Krikamol Muandet, Bharath K. Sriperumbudur

Publication date 17 April 2018

Full work available at URL https://arxiv.org/abs/1602.04361, http://jmlr.csail.mit.edu/papers/v18/17-032.html

zbMATH Keywords

reproducing kernel Hilbert spaceBochner integralBochner's theoremminimax lower boundstranslation invariant kernelkernel mean embeddings

Mathematics Subject Classification ID

Estimation in multivariate analysis (62H12) Minimax procedures in statistical decision theory (62C20) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22)

Abstract: In this paper, we study the minimax estimation of the Bochner integral ${\displaystyle }$mu_k(P):=int_{mathcal{X}} k(cdot,x),dP(x),${\displaystyle }$ also called as the kernel mean embedding, based on random samples drawn i.i.d.~from $P$, where $k:mathcalXimesmathcalXightarrowmathbbR$ is a positive definite kernel. Various estimators (including the empirical estimator), $hat{heta}_n$ of $m{u}_k(P)$ are studied in the literature wherein all of them satisfy $$ with $mathcal{H}_k$ being the reproducing kernel Hilbert space induced by $k$. The main contribution of the paper is in showing that the above mentioned rate of $n^{-1/2}$ is minimax in $|cdot{|}_{mathcal{H}_k}$ and $|cdot{|}_{L^2(mathbb{R}^d)}$-norms over the class of discrete measures and the class of measures that has an infinitely differentiable density, with $k$ being a continuous translation-invariant kernel on $mathbb{R}^d$. The interesting aspect of this result is that the minimax rate is independent of the smoothness of the kernel and the density of $P$ (if it exists). This result has practical consequences in statistical applications as the mean embedding has been widely employed in non-parametric hypothesis testing, density estimation, causal inference and feature selection, through its relation to energy distance (and distance covariance).

Recommendations

• Kernel mean shrinkage estimators
• Minimax estimation of the integral of a power of a density
• On the optimal estimation of probability measures in weak and strong topologies
• Kernel distribution embeddings: universal kernels, characteristic kernels and kernel metrics on distributions
• Density estimation by kernel and wavelets methods: optimality of Besov spaces

Cites work

• scientific article; zbMATH DE number 3576139 (Why is no real title available?)
• scientific article; zbMATH DE number 1220667 (Why is no real title available?)
• scientific article; zbMATH DE number 1324223 (Why is no real title available?)
• scientific article; zbMATH DE number 1528193 (Why is no real title available?)
• scientific article; zbMATH DE number 1540658 (Why is no real title available?)
• A Hilbert Space Embedding for Distributions
• A kernel two-sample test
• Consistency and convergence rates of one-class SVMs and related algorithms
• Feature selection via dependence maximization
• Hilbert space embeddings and metrics on probability measures
• Introduction to nonparametric estimation
• Kernel Bayes' rule: Bayesian inference with positive definite kernels
• Kernel mean shrinkage estimators
• Measure theory. Vol. I and II
• On the empirical estimation of integral probability metrics
• On the optimal estimation of probability measures in weak and strong topologies
• Real Analysis and Probability
• Scattered Data Approximation
• Sums and Gaussian vectors
• Support Vector Machines
• Theory of Reproducing Kernels
• Uniform Central Limit Theorems
• Universality, Characteristic Kernels and RKHS Embedding of Measures

Cited in

(11)

• Diffeomorphic measure matching with kernels for generative modeling
• Variance-aware estimation of kernel mean embedding
• On the optimal estimation of probability measures in weak and strong topologies
• Comparing a large number of multivariate distributions
• Kernel Mean Embedding of Distributions: A Review and Beyond
• scientific article; zbMATH DE number 7415105 (Why is no real title available?)
• Minimax rate of distribution estimation on unknown submanifolds under adversarial losses
• Efficient numerical integration in reproducing kernel Hilbert spaces via leverage scores sampling
• Model-based kernel sum rule: kernel Bayesian inference with probabilistic models
• Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
• Statistical depth meets machine learning: kernel mean embeddings and depth in functional data analysis

This page was built for publication: Minimax estimation of kernel mean embeddings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4636999)