Concentration Phenomenon for Random Dynamical Systems: An Operator Theoretic Approach

MaRDI QIDQ6419849zbMATH OpenFDO

Authors Muhammad Naeem, Miroslav Pajic

Publication date 7 December 2022

Abstract: Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$m{u}_{pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The main contribution of this paper is circumventing tedious probabilistic methods with a study of a composition of the Markov transition operator $P$ followed by a multiplication operator defined by $e^r$. It turns out that even if the observable/ reward function is unbounded, but for some for some $q>2$, $$ and $P$ is hyperbounded with norm control $|P{|}_{2ightarrowq}<e^{frac12[frac12-frac1q]}$, sharp non-asymptotic concentration bounds follow. emph{Transport-entropy} inequality ensures the aforementioned upper bound on multiplication operator for all $q>2$. The role of emph{reversibility} in concentration phenomenon is demystified. These results are particularly useful for the reinforcement learning and controls communities as they allow for concentration inequalities w.r.t standard unbounded obersvables/reward functions where exact knowledge of the system is not available, let alone the reversibility of stationary measure.

This page was built for publication: Concentration Phenomenon for Random Dynamical Systems: An Operator Theoretic Approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6419849)