Divergence rates of Markov order estimators and their application to statistical estimation of stationary ergodic processes

DOI10.3150/12-BEJ468zbMATH Open1412.62113arXiv1307.6338MaRDI QIDQ358133FDOQ358133

Authors: Zsolt Talata 

Publication date: 16 August 2013

Published in: Bernoulli (Search for Journal in Brave)

Abstract: Stationary ergodic processes with finite alphabets are estimated by finite memory processes from a sample, an n-length realization of the process, where the memory depth of the estimator process is also estimated from the sample using penalized maximum likelihood (PML). Under some assumptions on the continuity rate and the assumption of non-nullness, a rate of convergence in $$-distance is obtained, with explicit constants. The result requires an analysis of the divergence of PML Markov order estimators for not necessarily finite memory processes. This divergence problem is investigated in more generality for three information criteria: the Bayesian information criterion with generalized penalty term yielding the PML, and the normalized maximum likelihood and the Krichevsky-Trofimov code lengths. Lower and upper bounds on the estimated order are obtained. The notion of consistent Markov order estimation is generalized for infinite memory processes using the concept of oracle order estimates, and generalized consistency of the PML Markov order estimator is presented.

Full work available at URL: https://arxiv.org/abs/1307.6338

Recommendations

• The optimal error exponent for Markov order estimation
• The consistency of the BIC Markov order estimator.
• Order Estimation of Markov Chains
• scientific article; zbMATH DE number 4082788
• On the minimal penalty for Markov order estimation

zbMATH Keywords

penalized maximum likelihoodrate of convergenceinformation criteriaoracle inequalitiesminimum description lengthfinite memory estimatorinfinite memoryMarkov approximation

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Markov processes: estimation; hidden Markov models (62M05) Stationary stochastic processes (60G10) Strong limit theorems (60F15)

Cites Work

• Estimating the dimension of a model
• Ideal spatial adaptation by wavelet shrinkage
• Title not available (Why is that?)
• Elements of Information Theory
• Gaussian model selection
• Twice-universal coding
• Risk bounds for model selection via penalization
• Context tree estimation for not necessarily finite memory processes, via BIC and MDL
• Measure concentration for a class of random processes
• Speed of \(\overline d\)-convergence for Markov approximations of chains with complete connections. A coupling approach
• An application of ergodic theory to probability theory
• A new covariance inequality and applications.
• Title not available (Why is that?)
• Bandwidth selection in nonparametric kernel testing
• The minimum description length principle in coding and modeling
• New dependence coefficients. Examples and applications to statistics
• The consistency of the BIC Markov order estimator.
• Processes with long memory: Regenerative construction and perfect simulation
• Information theory. Coding theorems for discrete memoryless systems
• Title not available (Why is that?)
• Chains with infinite connections: Uniqueness and Markov representation
• Prediction of random sequences and universal coding
• Markov approximations of chains of infinite order
• Exponential inequalities for empirical unbounded context trees
• The performance of universal encoding
• Title not available (Why is that?)
• Fluctuations of the Empirical Entropies of a Chain of Infinite Order
• Large-scale typicality of Markov sample paths and consistency of MDL order estimators
• The optimal error exponent for Markov order estimation
• On Analytic Properties of Entropy Rate
• On Rate of Convergence of Statistical Estimation of Stationary Ergodic Processes
• Universal codes as a basis for time series testing
• On the minimal penalty for Markov order estimation
• Markov approximation and consistent estimation of unbounded probabilistic suffix trees
• How sampling reveals a process
• Some upper bounds for the rate of convergence of penalized likelihood context tree estimators

Cited In (1)

• Strongly consistent code-based identification and order estimation for constrained finite-state model classes

This page was built for publication: Divergence rates of Markov order estimators and their application to statistical estimation of stationary ergodic processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q358133)