Predictive rate-distortion for infinite-order Markov processes
From MaRDI portal
(Redirected from Publication:310017)
Abstract: Predictive rate-distortion analysis suffers from the curse of dimensionality: clustering arbitrarily long pasts to retain information about arbitrarily long futures requires resources that typically grow exponentially with length. The challenge is compounded for infinite-order Markov processes, since conditioning on finite sequences cannot capture all of their past dependencies. Spectral arguments show that algorithms which cluster finite-length sequences fail dramatically when the underlying process has long-range temporal correlations and can fail even for processes generated by finite-memory hidden Markov models. We circumvent the curse of dimensionality in rate-distortion analysis of infinite-order processes by casting predictive rate-distortion objective functions in terms of the forward- and reverse-time causal states of computational mechanics. Examples demonstrate that the resulting causal rate-distortion theory substantially improves current predictive rate-distortion analyses.
Recommendations
Cites work
- scientific article; zbMATH DE number 46194 (Why is no real title available?)
- scientific article; zbMATH DE number 107774 (Why is no real title available?)
- scientific article; zbMATH DE number 3623432 (Why is no real title available?)
- scientific article; zbMATH DE number 3371972 (Why is no real title available?)
- A Mathematical Theory of Communication
- A mapping approach to rate-distortion computation and analysis
- Bifurcation structure of a class of \(S_N\)-invariant constrained optimization problems
- Computational mechanics: pattern and prediction, structure and simplicity.
- Elements of Information Theory
- Exact complexity: the spectral decomposition of intrinsic computation
- Excess entropy in natural language: present state and perspectives
- Fluctuation spectroscopy
- Hidden Markov Models for Speech Recognition
- Hidden Markov processes
- INFORMATION BOTTLENECKS, CAUSAL STATES, AND STATISTICAL RELEVANCE BASES: HOW TO REPRESENT RELEVANT INFORMATION IN MEMORYLESS TRANSDUCTION
- Information anatomy of stochastic equilibria
- Information symmetries in irreversible processes
- Optimal causal inference: estimating stored information and approximating causal architecture
- Predictability, complexity, and learning
- Prediction, retrodiction, and the amount of information stored in the present
- Predictive Coding and the Slowness Principle: An Information-Theoretic Approach
- Properties of the statistical complexity functional and partially deterministic HMMs
- Regularities unseen, randomness observed: Levels of entropy convergence
- Symmetry Breaking in Soft Clustering Decoding of Neural Codes
- The calculi of emergence: Computation, dynamics and induction
- The computational structure of spike trains
Cited in
(9)- Discovering causal structure with reproducing-kernel Hilbert space \(\epsilon\)-machines
- On asymptotically optimal methods of prediction and adaptive coding for Markov sources
- Shannon entropy rate of hidden Markov processes
- Rate distortion function for Nth-order Gaussian Markov process
- Correction to: ``Predictive rate-distortion for infinite-order Markov processes
- Predictive information in Gaussian processes with application to music analysis
- Topology, convergence, and reconstruction of predictive states
- Prediction and dissipation in nonequilibrium molecular sensors: conditionally Markovian channels driven by memoryful environments
- Spectral simplicity of apparent complexity. I: The nondiagonalizable metadynamics of prediction
This page was built for publication: Predictive rate-distortion for infinite-order Markov processes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q310017)
