Regularities unseen, randomness observed: Levels of entropy convergence
From MaRDI portal
(Redirected from Publication:5706339)
Abstract: We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to stochastic and deterministic processes by using successive derivatives of the Shannon entropy growth curve. This leads, in turn, to natural measures of apparent memory stored in a source and the amounts of information that must be extracted from observations of a source in order for it to be optimally predicted and for an observer to synchronize to it. One consequence of ignoring these structural properties is that the missed regularities are converted to apparent randomness. We demonstrate that this problem arises particularly for small data sets; e.g., in settings where one has access only to short measurement sequences.
Recommendations
- Absolutely regular empirical processes and universal entropy
- Random entropy and recurrence
- Entropy Via Random Perturbations
- Random irreversible phenomena: Entropy in subordination
- Some observations on the concepts of information-theoretic entropy and randomness
- Entropy in uniformly quasiregular dynamics
- Entropy, Randomization, Derandomization, and Discrepancy
- Entropy versus level sets
- Possible rates of entropy convergence
Cites work
- A Mathematical Theory of Communication
- Characteristic quantities of multifractals -- application to the Feigenbaum attractor
- Complexity of two-dimensional patterns
- Complexity of unimodal maps with aperiodic kneading sequences
- Computational mechanics: pattern and prediction, structure and simplicity.
- Conditional entropy of some automatic sequences
- Entropy estimation of symbol sequences
- Entropy of the symbolic sequence for critical circle maps.
- ENTROPY, TRANSINFORMATION AND WORD DISTRIBUTION OF INFORMATION-CARRYING SEQUENCES
- Fluctuation spectroscopy
- Measures of statistical complexity: why?
- Mutual information functions versus correlation functions.
- On the Identifiability Problem for Functions of Finite Markov Chains
- Power law distributions of spectral density and higher order entropies
- Predictability, complexity, and learning
- Prediction and entropy of nonlinear dynamical systems and symbolic sequences with LRO
- Queues, stacks, and transcendentality at the transition to chaos
- Statistical properties of chaos demonstrated in a class of one-dimensional maps
- Structural Information in Self-Organizing Systems
- Subshifts of finite type and sofic systems
- Symbolic dynamics of one-dimensional maps: Entropies, finite precision, and noise
- The calculi of emergence: Computation, dynamics and induction
- Toward a quantitative theory of self-generated complexity
Cited in
(62)- Relative entropy measures applied to healthy and pathological voice characterization
- A general definition of conditional information and its application to ergodic decomposition
- Prediction, retrodiction, and the amount of information stored in the present
- Correlated biased random walk with latency in one and two dimensions: asserting patterned and unpredictable movement
- The approach towards equilibrium in a reversible Ising dynamics model: an information-theoretic analysis based on an exact solution
- Predictive models and generative complexity
- Shannon entropy rate of hidden Markov processes
- On hidden Markov processes with infinite excess entropy
- Exact complexity: the spectral decomposition of intrinsic computation
- Statistical signatures of structural organization: the case of long memory in renewal processes
- On processes with summable partial autocorrelations
- The ambiguity of simplicity in quantum and classical simulation
- Leveraging environmental correlations: the thermodynamics of requisite variety
- Informational and causal architecture of continuous-time renewal processes
- Predictive information in a nonequilibrium critical model
- Stability and diversity in collective adaptation
- Topology, convergence, and reconstruction of predictive states
- Permutation complexity and coupling measures in hidden Markov models
- Pairwise correlations in layered close-packed structures
- Information anatomy of stochastic equilibria
- Predictive rate-distortion for infinite-order Markov processes
- A METHOD FOR INFERRING HIERARCHICAL DYNAMICS IN STOCHASTIC PROCESSES
- Optimal instruments and models for noisy chaos
- Information accessibility and cryptic processes
- Spectral simplicity of apparent complexity. I: The nondiagonalizable metadynamics of prediction
- Spectral simplicity of apparent complexity. II: Exact complexities and complexity spectra
- Closure measures for coarse-graining of the tent map
- A revision of coding theory for learning from language
- Extrinsic faulting in \(3C\) close-packed crystal structures: computational mechanics analysis
- Quantum cognition and bounded rationality
- The mathematical structure of the genetic code: a tool for inquiring on the origin of life
- Synchronization and control in intrinsic and designed computation: An information-theoretic analysis of competing models of stochastic computation
- Information modification and particle collisions in distributed computation
- Optimal causal inference: estimating stored information and approximating causal architecture
- Nature computes: information processing in quantum dynamical systems
- A geometric approach to complexity
- Excess entropy in natural language: present state and perspectives
- Information symmetries in irreversible processes
- Anatomy of a bit: information in a time series observation
- How hidden are hidden processes? A primer on crypticity and entropy convergence
- Local entropy and structure in a two-dimensional frustrated system
- SYNCHRONIZING TO PERIODICITY: THE TRANSIENT INFORMATION AND SYNCHRONIZATION TIME OF PERIODIC SEQUENCES
- Computational capabilities at the edge of chaos for one dimensional systems undergoing continuous transitions
- \(\epsilon\)-machine spectral reconstruction theory: a direct method for inferring planar disorder and structure from X-ray diffraction studies
- Intrinsic quantum computation
- Generalized conditional entropy -- determinicity of a process and Rokhlin's formula
- Identifying functional thermodynamics in autonomous Maxwellian ratchets
- An informatic approach to a long memory stationary process
- Optimality and complexity in measured quantum-state stochastic processes
- Correlation and collective behaviour in Adler-type locally coupled oscillators at the edge of chaos
- Partially ordered permutation entropies
- Symbolic transfer entropy rate is equal to transfer entropy rate for bivariate finite-alphabet stationary ergodic Markov processes
- Divergent predictive states: the statistical complexity dimension of stationary, ergodic hidden Markov processes
- Exploring predictive states via Cantor embeddings and Wasserstein distance
- Exact synchronization for finite-state sources
- Asymptotic synchronization for finite-state sources
- Information theory for complex systems scientists: what, why, and how
- Permutation complexity via duality between values and orderings
- Exponential bounds for convergence of entropy rate approximations in hidden Markov models satisfying a path-mergeability condition
- Computation in finitary stochastic and quantum processes
- Variable-length coding of two-sided asymptotically mean stationary measures
- How should complexity scale with system size?
This page was built for publication: Regularities unseen, randomness observed: Levels of entropy convergence
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5706339)
