Fisher-Rao gradient flows of linear programs and state-action natural policy gradients

DOI10.1137/24M1653422MaRDI QIDQ6974653zbMATH OpenFDO

Authors Johannes Müller, Semih Cayci, Guido Montúfar

Publication date 5 June 2025

Published in SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1137/24M1653422

zbMATH Keywords

entropic regularizationFisher-Rao metriclinear programMarkov decision processmultiplayer gamenatural policy gradient

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Linear programming (90C05)

Cites work

• Algebraic foundation of mathematical statistics²
• Approximate Newton Policy Gradient Algorithms
• Asymptotic analysis of the exponential penalty trajectory in linear programming
• Computational optimal transport. With applications to data sciences
• Convergence of entropy-regularized natural policy gradient with linear function approximation
• Fast global convergence of natural policy gradient methods with entropy regularization
• Finding the Maximizers of the Information Divergence From an Exponential Family
• Finite state Markovian decision processes
• Geometry and convergence of natural policy gradient methods
• Global optimality guarantees for policy gradient methods
• Hessian informed mirror descent
• Hessian Riemannian Gradient Flows in Convex Programming
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 3106666 (Why is no real title available?)
• Information geometry
• Information geometry and its applications
• Invariance properties of the natural gradient in overparametrised systems
• On linear and super-linear convergence of natural policy gradient algorithm
• On linear convergence of non-Euclidean gradient methods without strong convexity and Lipschitz gradient continuity
• On the Fisher metric of conditional probability polytopes
• On the theory of policy gradient methods: optimality, approximation, and distribution shift
• Ordinary differential equations and dynamical systems
• Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence
• Policy mirror descent for reinforcement learning: linear convergence, new sampling complexity, and generalized problem classes
• Relatively smooth convex optimization by first-order methods, and applications
• Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory
• Toric geometry of entropic regularization

Cited in

(1)

• Optimal rates of convergence for entropy regularization in discounted Markov decision processes

This page was built for publication: Fisher-Rao gradient flows of linear programs and state-action natural policy gradients

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6974653)