Thermodynamics and concentration (Q418227): Difference between revisions

Concentration inequalities bound the probability that random quantities deviate from their average, medicine or otherwise typical value. They play an important role in the study of natural and artificial learning systems. The aim of this paper is to introduce the subadditivity of entropy as a unified basis for the derivation of concentration inequalities for functions on product spaces and to demonstrate the benefits of formulations for the concentration problem in the language of statistical thermodynamic. The author's method consists of three steps. In the first step, he expresses the log-Laplace transform in the terms of an integral of the thermal entropy over a range of inverse temperature. In the second step, he proves the tensorization inequality or, more precisely speaking, a thermal subadditivity property of entropy. It is proved that the entropy of a system is not greater than the thermal average of the sum of entropies of the constituent subsystems. In the third step, he expresses the entropy of the subsystem in terms of thermal energy fluctuations. At the end, he shows some applications of his method.