A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning

DOI10.1214/16-AOS1465MaRDI QIDQ2012203zbMATHOpenAlexFDO

Authors Hanyuan Hang, Ingo Steinwart

Publication date 28 July 2017

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1501.03059

zbMATH Keywords

dynamical systemsBernstein-type inequalitiessupport vector machines (SVMs)mixing processesnonparametric classification and regression

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Learning and adaptive systems in artificial intelligence (68T05) Large deviations (60F10) Inequalities; stochastic orderings (60E15) Stationary stochastic processes (60G10) Uniformly hyperbolic systems (expanding, Anosov, Axiom A, etc.) (37D20)

Abstract: We establish a Bernstein-type inequality for a class of stochastic processes that include the classical geometrically $phi$-mixing processes, Rio's generalization of these processes, as well as many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d.~data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support vector machines using the Gaussian kernels for both least squares and quantile regression, it turns out that the resulting learning rates match, up to some arbitrarily small extra term in the exponent, the optimal rates for i.i.d.~processes.

Recommendations

• Learning theory estimates with observations from general stationary stochastic processes
• New Bernstein's inequalities for dependent observations and applications to learning theory
• Fast learning from \(\alpha\)-mixing observations
• An oracle inequality for regularized risk minimizers with strongly mixing observations
• Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods

Cited in

(18)

• A Bernstein-type inequality for high dimensional linear processes with applications to robust estimation of time series regressions
• Robbins-Monro algorithm with \(\psi\)-mixing random errors
• Empirical risk minimization and complexity of dynamical models
• Structure learning via unstructured kernel-based M-estimation
• Dynamic Matrix Recovery
• Kernel density estimation for dynamical systems
• Recovery guarantees for polynomial coefficients from weakly dependent data with outliers
• Gibbs posterior convergence and the thermodynamic formalism
• An oracle inequality for regularized risk minimizers with strongly mixing observations
• Fast learning from \(\alpha\)-mixing observations
• Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods
• Exponential inequalities for nonstationary Markov chains
• Learning theory estimates with observations from general stationary stochastic processes
• Concentration inequalities for empirical processes of linear time series
• On the need for structure modelling in sequence prediction
• Local linear regression with nonparametrically generated covariates for weakly dependent data
• An extension of Bernstein inequality
• New Bernstein's inequalities for dependent observations and applications to learning theory

This page was built for publication: A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2012203)