From -entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
From MaRDI portal
From \(\varepsilon\)-entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
Abstract: We consider an extension of -entropy to a KL-divergence based complexity measure for randomized density estimation methods. Based on this extension, we develop a general information-theoretical inequality that measures the statistical complexity of some deterministic and randomized density estimators. Consequences of the new inequality will be presented. In particular, we show that this technique can lead to improvements of some classical results concerning the convergence of minimum description length and Bayesian posterior distributions. Moreover, we are able to derive clean finite-sample convergence bounds that are not obtainable using previous approaches.
Recommendations
- The complexity of estimating min-entropy
- k_n-nearest neighbor estimators of entropy
- Entropic measures, Markov information sources and complexity
- On information gain, Kullback-Leibler divergence, entropy production and the involution kernel
- Some further results on the minimum error entropy estimation
- Comparison of maximum entropy and minimal mutual information in a nonlinear setting
- The complexity of estimating Rényi entropy
- Unifying computational entropies via Kullback-Leibler divergence
- Sublinear estimation of entropy and information distances
Cites work
- 10.1162/1532443041424300
- Asymptotic methods in statistical decision theory
- Convergence of estimates under dimensionality restrictions
- Convergence rates of posterior distributions.
- scientific article; zbMATH DE number 3173999 (Why is no real title available?)
- scientific article; zbMATH DE number 45100 (Why is no real title available?)
- scientific article; zbMATH DE number 1420699 (Why is no real title available?)
- Information-theoretic determination of minimax rates of convergence
- Minimum complexity density estimation
- On Bayesian consistency
- PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification
- Rates of convergence of posterior distributions.
- The consistency of posterior distributions in nonparametric problems
- Weak convergence and empirical processes. With applications to statistics
Cited in
(50)- Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
- Sparse recovery in convex hulls via entropy penalization
- Information-theoretic determination of minimax rates of convergence
- Bayesian fractional posteriors
- Fast adaptive estimation of log-additive exponential models in Kullback-Leibler divergence
- Approximate models and robust decisions
- Contextuality of misspecification and data-dependent losses
- Dynamics of Bayesian updating with dependent data and misspecified models
- Adaptive Bayesian density estimation with location-scale mixtures
- PAC-Bayesian bounds for sparse regression estimation with exponential weights
- Optimal rates of entropy estimation over Lipschitz balls
- Gibbs posterior inference on multivariate quantiles
- Gibbs posterior convergence and the thermodynamic formalism
- Model-free posterior inference on the area under the receiver operating characteristic curve
- New estimates for Csiszár divergence and Zipf-Mandelbrot entropy via Jensen-Mercer's inequality
- On general Bayesian inference using loss functions
- Quasi-Bayesian analysis of nonparametric instrumental variables models
- Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
- Posterior concentration and fast convergence rates for generalized Bayesian learning
- Consistency and generalization bounds for maximum entropy density estimation
- Predicting Panel Data Binary Choice with the Gibbs Posterior
- Kolmogrov's -entropy and the problem of statistical estimation
- Minimum complexity density estimation
- A Justification for Applying the Principle of Minimum Relative Entropy to Information Integration Problems
- Mirror averaging with sparsity priors
- Learning Theory
- Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector
- Fast rates for general unbounded loss functions: from ERM to generalized Bayes
- Chaining meets chain rule: multilevel entropic regularization and training of neural networks
- Minimum description length revisited
- scientific article; zbMATH DE number 7625184 (Why is no real title available?)
- Model misspecification, Bayesian versus credibility estimation, and Gibbs posteriors
- Gibbs posterior inference on value-at-risk
- Generalized Bayes Quantification Learning under Dataset Shift
- A comparison of learning rate selection methods in generalized Bayesian inference
- Joint production in stochastic non-parametric envelopment of data with firm-specific directions
- Adaptive variational Bayes: optimality, computation and applications
- Adaptive variable selection for sequential prediction in multivariate dynamic models
- Approximating Bayes in the 21st century
- Heavy-tailed Bayesian nonparametric adaptation
- Offline reinforcement learning in large state spaces: algorithms and guarantees
- Deep horseshoe Gaussian processes
- Bayesian design principles for frequentist sequential learning
- Loss-Based Variational Bayes Prediction
- A tight excess risk bound via a unified PAC-Bayesian-Rademacher-Shtarkov-MDL complexity
- Sequential Gibbs posteriors with applications to principal component analysis
- Generalized mirror averaging and D-convex aggregation
- Linear and convex aggregation of density estimators
- Learning by mirror averaging
- Gibbs posterior for variable selection in high-dimensional classification and data mining
This page was built for publication: From \(\varepsilon\)-entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q869967)
