Consistency and generalization bounds for maximum entropy density estimation

DOI10.3390/E15125439MaRDI QIDQ280753zbMATH OpenOpenAlexFDO

Authors Shaojun Wang, Russell Greiner, Shaomin Wang

Publication date 10 May 2016

Published in Entropy (Search for Journal in Brave)

Full work available at URL https://doi.org/10.3390/e15125439

zbMATH Keywords

consistencydensity estimationgeneralization boundmaximum entropy principle

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Density estimation (62G07)

Review text

Summary: We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity of the underlying feature functions. This allows us to establish the universal consistency of maximum entropy density estimation.

Recommendations

• Learning Theory
• Maximum Entropy Distribution Estimation with Generalized Regularization
• Maximum entropy density estimation with generalized regularization and an application to species distribution modeling
• From -entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
• Large sample properties of maximum entropy histograms

Cites work

• 10.1162/153244302760200713
• 10.1162/1532443041424300
• A mathematical theory of communication
• Approximation of density functions by sequences of exponential families
• Boosting with early stopping: convergence and consistency
• Convex analysis and nonlinear optimization. Theory and examples
• Empirical margin distributions and bounding the generalization error of combined classifiers
• Graphical models, exponential families, and variational inference
• scientific article; zbMATH DE number 4170917 (Why is no real title available?)
• scientific article; zbMATH DE number 5654889 (Why is no real title available?)
• scientific article; zbMATH DE number 49190 (Why is no real title available?)
• scientific article; zbMATH DE number 1220667 (Why is no real title available?)
• I-divergence geometry of probability distributions and minimization problems
• Information Theory and Statistical Mechanics
• Leave-One-Out Bounds for Kernel Methods
• Risk bounds for mixture density estimation
• Statistical behavior and consistency of classification methods based on convex risk minimization.
• The EM Algorithm and Extensions, 2E
• Uniform Central Limit Theorems
• Weak convergence and empirical processes. With applications to statistics

Cited in

(10)

• Maximum-entropy inference and inverse continuity of the numerical range
• Relative entropy derivative bounds
• Convexity inequalities for estimating generalized conditional entropies from below
• Maximum entropy density estimation with generalized regularization and an application to species distribution modeling
• Using Maximum Entropy and Generalized Belief Propagation in Estimation of Distribution Algorithms
• Learning Theory
• Maximum entropy in the mean methods in propensity score matching for interval and noisy data
• Maximum Entropy Distribution Estimation with Generalized Regularization
• Nonparametric e-mixture estimation
• Minimax rates for conditional density estimation via empirical entropy

This page was built for publication: Consistency and generalization bounds for maximum entropy density estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q280753)