On some entropy functionals derived from Rényi information divergence

DOI10.1016/J.INS.2008.02.003MaRDI QIDQ1031673zbMATH OpenOpenAlexFDO

Authors J-F Bercher

Publication date 30 October 2009

Published in Information Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0805.0129

zbMATH Keywords

maximum entropy principlenonextensivityRényi divergencesRényi entropyTsallis distributions

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Abstract: We consider the maximum entropy problems associated with R'enyi $Q$-entropy, subject to two kinds of constraints on expected values. The constraints considered are a constraint on the standard expectation, and a constraint on the generalized expectation as encountered in nonextensive statistics. The optimum maximum entropy probability distributions, which can exhibit a power-law behaviour, are derived and characterized. The R'enyi entropy of the optimum distributions can be viewed as a function of the constraint. This defines two families of entropy functionals in the space of possible expected values. General properties of these functionals, including nonnegativity, minimum, convexity, are documented. Their relationships as well as numerical aspects are also discussed. Finally, we work out some specific cases for the reference measure $Q(x)$ and recover in a limit case some well-known entropies.

Recommendations

• Some results concerning maximum Rényi entropy distributions
• Rescuing the MaxEnt treatment for \(q\)-generalized entropies
• On scale and concentration invariance in entropies.
• scientific article; zbMATH DE number 5263257
• Probability distributions and the maximum entropy principle

Cites work

• A generalized divergence measure for robust image registration
• Clustering with Bregman divergences.
• Comparison of maximum entropy and higher-order entropy estimators.
• Divergence statistics based on entropy functions and stratified sampling
• Dual description of nonextensive ensembles
• Entropic nonextensivity: A possible measure of complexity
• Expressions for Rényi and Shannon entropies for bivariate distributions
• Formulas for Rényi information and related measures for univariate distributions.
• Generalized cutoff rates and Renyi's information measures
• Generalized Pareto Fit to the Society of Actuaries’ Large Claims Database
• scientific article; zbMATH DE number 3173999 (Why is no real title available?)
• scientific article; zbMATH DE number 3903723 (Why is no real title available?)
• scientific article; zbMATH DE number 3241743 (Why is no real title available?)
• Image matching using alpha-entropy measures and entropic graphs
• Information Theory and Statistical Mechanics
• Maximum probability/entropy translating of contiguous categorical observations into frequencies
• Measuring time-frequency information content using the Renyi entropies
• Possible generalization of Boltzmann-Gibbs statistics.
• Rényi continuous entropy of DNA sequences
• Rényi information measure for a used item
• Rényi information, loglikelihood and an intrinsic distribution measure
• Rényi statistics for testing hypotheses in mixed linear regression models
• Renyi's entropy as an index of diversity in simple-stage cluster sampling
• Some new information measures for fuzzy sets
• Study of DNA binding sites using the Rényi parametric entropy measure
• The world according to Rényi: Thermodynamics of multifractal systems
• Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems

Cited in

(22)

• On scale and concentration invariance in entropies.
• Geometric characterization of Weyl's discrepancy norm in terms of its \(n\)-dimensional unit balls
• The \(q\)-exponentials do not maximize the Rényi entropy
• Rényi relative entropy from homogeneous Kullback-Leibler divergence Lagrangian
• Rényi entropy rate for Gaussian processes
• Some properties of Rényi entropy and Rényi entropy rate
• On uncertainty and information properties of ranked set samples
• Renyi entropy properties of mixed systems
• scientific article; zbMATH DE number 124792 (Why is no real title available?)
• Maximum entropy approach to stretched exponential probability distributions
• A Variational Characterization of Rényi Divergences
• A Scale-Invariant Generalization of the Rényi Entropy, Associated Divergences and Their Optimizations Under Tsallis’ Nonextensive Framework
• On Smooth Rényi Entropies: A Novel Information Measure, One-Shot Coding Theorems, and Asymptotic Expansions
• A Matlab program to calculate distribution with maximum Renyi entropy
• ON THE PROBLEM OF CONSTRAINTS IN NONEXTENSIVE FORMALISM: A QUANTUM MECHANICAL TREATMENT
• Refined estimations for some types of entropies and divergences
• Analyzing supersaturated designs with entropic measures
• The Pólya information divergence
• Results on residual Rényi entropy of order statistics and record values
• Divergence statistics for testing uniform association in cross-classifications
• Escort entropies and divergences and related canonical distribution
• Some results concerning maximum Rényi entropy distributions

This page was built for publication: On some entropy functionals derived from Rényi information divergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1031673)