No label defined (Q4946941)

The entropy functional \[ G(\{p_{i}\})=-\sum_{i}p_{i}\ln p_{i} \] is derived as the unique maximand for assigning probabilities given linear constraints. This new derivation presents the principle of maximum entropy for assigning probability distributions. Many derivations of this form exist: these include the Shannon-Jaynes argument, discussed in this paper; a combinatorial argument, which uses Stirling's approximation in the limit of a large, possibly imaginary, ensemble; and the Shore-Johnson argument that a joint distribution to be reconstructed from its single-variable marginals should not exhibit correlation between the variables, since no evidence is given for it. All these arguments give the same result but begin from different axioms, so that which derivation you prefer depends simply on which starting axioms you prefer. In common with the Shannon-Jaynes derivation the idea involves an invariance under arbitrary partitioning of the points on which probabilities are defined. However, the Shannon-Jaynes derivation is a two-stage process: asserting that a particular functional of the probabilities, that has a certain interpretation, should be maximized, and then using that interpretation to generate an equation for the functional itself. This equation is routinely solved to give the usual \(p\ln p\) form. By contrast, the derivation presented in this paper is a one-stage process which does not rest on any interpretation. For the entire collection see [Zbl 0928.00040].