Redistributing algorithms and Shannon's entropy (Q2123707)

Shannon's entropy is one of the most popular measures of disorder. Together with the concept of permutation entropy, they are used to quantify the uncertainty and disorder of a time series. This is based on the appearance of ordinal patterns. There are several approaches to calculate the tuples rearrangement of the components with the ups and downs. This is used for instance in the analysis of experimental fire data. However, depending on data features it can be required to redistribute the rare/less encountered tuples. But it is not known how the redistributing algorithm (used when some of the less encountered \(j\)-tuples are ignored/redistributed) affects the calculations and values of permutation entropy. The authors stated an open problem of finding a lower bound \(a(k)\ge 0\) for probability distribution \(P\) that has at least \(k\) nonzero and equal components \(\ge a(k)\) and the Shannon entropy \(H(P)\) reaches its minimum when \(n-k\) components of \(P\) are zero. They give a solution to this by studying how a redistributing algorithm affects the entropy and propose a final solution to the stated problem.