Minimax Optimal Estimation of KL Divergence for Continuous Distributions
From MaRDI portal
Abstract: Estimating Kullback-Leibler divergence from identical and independently distributed samples is an important problem in various domains. One simple and effective estimator is based on the k nearest neighbor distances between these samples. In this paper, we analyze the convergence rates of the bias and variance of this estimator. Furthermore, we derive a lower bound of the minimax mean square error and show that kNN method is asymptotically rate optimal.
Recommendations
- Estimation of KL Divergence: Optimal Minimax Rate
- Minimization of the Kullback-Leibler divergence over a log-normal exponential arc
- A Minimax Result for f-Divergences
- Minimum \(f\)-divergence estimators and quasi-likelihood functions
- Density estimation with minimization of U-divergence
- The minimum S-divergence estimator under continuous models: the Basu-Lindsay approach
- scientific article; zbMATH DE number 1966630
- Bounds for Kullback-Leibler divergence
- Minimum divergence estimators, maximum likelihood and exponential families
Cited in
(5)- Derivation of mixture distributions and weighted likelihood function as minimizers of KL-divergence subject to constraints
- Estimation of KL Divergence: Optimal Minimax Rate
- Distributionally robust trade‐off design of parity relation based fault detection systems
- Expanding the prediction capacity in long sequence time-series forecasting
- Embedding distributional data
This page was built for publication: Minimax Optimal Estimation of KL Divergence for Continuous Distributions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5138933)
