A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning

DOI10.1214/16-AOS1465zbMATH Open1388.60060arXiv1501.03059OpenAlexW2963245453MaRDI QIDQ2012203FDOQ2012203

Ingo Steinwart, Hanyuan Hang

Publication date: 28 July 2017

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We establish a Bernstein-type inequality for a class of stochastic processes that include the classical geometrically $phi$-mixing processes, Rio's generalization of these processes, as well as many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d.~data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support vector machines using the Gaussian kernels for both least squares and quantile regression, it turns out that the resulting learning rates match, up to some arbitrarily small extra term in the exponent, the optimal rates for i.i.d.~processes.

Full work available at URL: https://arxiv.org/abs/1501.03059

Recommendations

• Learning theory estimates with observations from general stationary stochastic processes
• New Bernstein's inequalities for dependent observations and applications to learning theory
• Fast learning from \(\alpha\)-mixing observations
• An oracle inequality for regularized risk minimizers with strongly mixing observations
• Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods

zbMATH Keywords

dynamical systemsBernstein-type inequalitiessupport vector machines (SVMs)mixing processesnonparametric classification and regression

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Learning and adaptive systems in artificial intelligence (68T05) Large deviations (60F10) Inequalities; stochastic orderings (60E15) Stationary stochastic processes (60G10) Uniformly hyperbolic systems (expanding, Anosov, Axiom A, etc.) (37D20)

Cited In (15)

• Recovery guarantees for polynomial coefficients from weakly dependent data with outliers
• Exponential inequalities for nonstationary Markov chains
• Empirical risk minimization and complexity of dynamical models
• Structure learning via unstructured kernel-based M-estimation
• On the need for structure modelling in sequence prediction
• An extension of Bernstein inequality
• Learning Theory Estimates with Observations from General Stationary Stochastic Processes
• Title not available (Why is that?)
• Local linear regression with nonparametrically generated covariates for weakly dependent data
• Dynamic Matrix Recovery
• Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods
• A Bernstein-type inequality for high dimensional linear processes with applications to robust estimation of time series regressions
• Title not available (Why is that?)
• Gibbs posterior convergence and the thermodynamic formalism
• Robbins-Monro algorithm with \(\psi\)-mixing random errors

This page was built for publication: A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2012203)