Inverse optimal control for averaged cost per stage linear quadratic regulators

DOI10.1016/J.SYSCONLE.2023.105658arXiv2305.15332OpenAlexW4388712152MaRDI QIDQ6131460FDOQ6131460

Authors:

Publication date: 5 April 2024

Published in: Systems \& Control Letters (Search for Journal in Brave)

Abstract: Inverse Optimal Control (IOC) is a powerful framework for learning a behaviour from observations of experts. The framework aims to identify the underlying cost function that the observed optimal trajectories (the experts' behaviour) are optimal with respect to. In this work, we considered the case of identifying the cost and the feedback law from observed trajectories generated by an ``average cost per stage" linear quadratic regulator. We show that identifying the cost is in general an ill-posed problem, and give necessary and sufficient conditions for non-identifiability. Moreover, despite the fact that the problem is in general ill-posed, we construct an estimator for the cost function and show that the control gain corresponding to this estimator is a statistically consistent estimator for the true underlying control gain. In fact, the constructed estimator is based on convex optimization, and hence the proved statistical consistency is also observed in practice. We illustrate the latter by applying the method on a simulation example from rehabilitation robotics.

Full work available at URL: https://arxiv.org/abs/2305.15332

Recommendations

• Statistically consistent inverse optimal control for discrete-time indefinite linear-quadratic systems
• Inverse optimal control for discrete-time finite-horizon linear quadratic regulators
• Continuous-time inverse quadratic optimal control problem
• Inverse linear-quadratic discrete-time finite-horizon optimal control for indistinguishable homogeneous agents: a convex optimization approach
• Linear conic optimization for inverse optimal control

zbMATH Keywords

convex optimizationsemidefinite programmingsystem identificationinverse optimal controlinverse reinforcement learning

Mathematics Subject Classification ID

Convex programming (90C25) Semidefinite programming (90C22) Inverse problems in optimal control (49N45) System identification (93B30)

Cites Work

• CVXPY: a Python-embedded modeling language for convex optimization
• Asymptotic Statistics
• Linear Matrix Inequalities in System and Control Theory
• Asymptotic Properties of Non-Linear Least Squares Estimators
• Graph implementations for nonsmooth convex programs
• Dynamic programming and optimal control. Vol. 1.
• Reinforcement learning. An introduction
• Finite-horizon inverse optimal control for discrete-time nonlinear systems
• A new approach to the LQ design from the viewpoint of the inverse regulator problem
• Inverse Problem of Linear Optimal Control
• Inverse optimal control for discrete-time finite-horizon linear quadratic regulators
• Inverse optimization with noisy data
• On the discrete time algebraic Riccati equation
• Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons
• Inverse linear-quadratic discrete-time finite-horizon optimal control for indistinguishable homogeneous agents: a convex optimization approach

Cited In (1)

• Statistically consistent inverse optimal control for discrete-time indefinite linear-quadratic systems

This page was built for publication: Inverse optimal control for averaged cost per stage linear quadratic regulators

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6131460)