scientific article; zbMATH DE number 4005215
1) This paper is the first in a series dedicated to the presentation of a method of best approximation of a given multi-dimensional probability measure with the help of probability measures belonging to some class. This method gives explicitly the best approximating element in the considered class as well as the expression of the corresponding least error. This least error is an information functional representing the amount of information determined by the given probability measure with respect to the considered class. 2) Let (X,S) be a measurable space; L the convex set of probability measures on (X,S); A an idempotent operator, defined on L with values in \(L'=A(L)\subset L\). If U,V\(\in L\), let h(U:V) be the relative entropy of U with respect to V. A is defined as the projector on L' and AP\(\in L'\) as the projection of \(P\in L\) on L'. The quantities \(I_ A(P)=h(P:AP)\), \(I^*_ A(P)=h(AP:P)\) are defined as the amount of A-information resp. of conjugate A-information determined by \(P\in L\). Let \(\hat P\) be the Fourier transform of \(P\in L\) and \(\| \cdot \|\) the total variation of a completely additive function defined on (X,S). We present here the main results of the paper. Theorem 1. If \(P\in L\), then \(I_ A(P)\geq 0\), \(A^*_ A(P)\geq 0\) with equalities iff \(P=AP\), i.e. if \(P\in L\). Theorem 2. In order that \(I_ A(P)\) resp. \(I^*_ A(P)\) be finite it is necessary that \(P\ll AP\), resp. \(AP\ll P\); in this case \(I_ A(P)\) is the integral on P over X of log[P(dx):(AP)(dx)] and \(I^*_ A(P)\) is the integral on AP over X of log[(AP)(dx):P(dx)]. Theorem 3. Under some regularity condition concerning A, if \(I_ A(P)\) is finite, then min\(\{\) h(P:Q); \(Q\in L'\}=I_ A(P)\) and this minimum is reached iff \(Q=AP\). Theorem 4. sup\(\{| \hat P(t)-(A\hat P)(t)|^ 2;t\}\leq \| P-AP\|^ 2\leq 2 \min \{I_ A(P),I^*_ A(P)\}\), i.e. if \(I_ A(P)\) or \(I^*_ A(P)\) is small, then P, AP are close in variation. 3) The paper presents the results of the approximation in the case L' consists of (a) independent sequences, (b) Markov chains of order \(r\geq 1\), (c) independent identically distributed sequences, (d) homogeneous Markov chains of order \(r\geq 1\). It presents also relations between the amounts of information corresponding to various situations. 4) Let \((X,S)=(X_ 1,S_ 1)\times (X_ 2,S_ 2)\); L, \(L_ 1\), \(L_ 2\) the sets of probability measures P, \(P_ 1\), \(P_ 2\) on (X,S), \((X_ 1,S_ 1)\), \((X_ 2,S_ 2)\) respectively; L' the set of probability measures \(P_ 1\times P_ 2\) in L; \(P^{(1)}\in L_ 1\), \(P^{(2)}\in L_ 2\) the marginals of \(P\in L\); A the operator \(P\to P^{(1)}\times P^{(2)}\). Then \(I_ A(P)\) is Shannon's amount of information about \(P^{(1)}\) contained in \(P^{(2)}\) and conversely. So, as a by-product of the approximation problem discussed, the paper offers a reasonable meaning of Shannon's amount of information (and simultaneously, of the concept of entropy, which is essentially equivalent to it): each time it is used, in reality a best approximation takes place. The method presented enlarges in a natural way Shannon's concept to a universal concept of information, taking as many particular forms as there are particular approximation problems.
- Approximation of nonlinear integral functionals
- Adjustment by minimum discriminant information
- Asymptotics of one-dimensional Lévy approximations
- Approximations of nonlinear integral functionals of entropy type
- scientific article; zbMATH DE number 3131443 (Why is no real title available?)
- scientific article; zbMATH DE number 3948415 (Why is no real title available?)
- On the relationship between the information measures and the Bayes probability of error
- scientific article; zbMATH DE number 4084718 (Why is no real title available?)
- scientific article; zbMATH DE number 3601518 (Why is no real title available?)
- scientific article; zbMATH DE number 4122932 (Why is no real title available?)
- scientific article; zbMATH DE number 3892353 (Why is no real title available?)
- Approximation in the space of information channels
- Information cost functions
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3756198)