Optimal control of port-Hamiltonian systems: a continuous-time learning approach

DOI10.1016/J.AUTOMATICA.2021.109725zbMATH Open1478.93320arXiv2007.08645OpenAlexW3166890310WikidataQ114204893 ScholiaQ114204893MaRDI QIDQ2665352FDOQ2665352

Authors: Lukas Kölsch, Pol Jané Soneira, Felix Strehle, Sören Hohmann 

Publication date: 19 November 2021

Published in: Automatica (Search for Journal in Brave)

Abstract: Feedback controllers for port-Hamiltonian systems reveal an intrinsic inverse optimality property since each passivating state feedback controller is optimal with respect to some specific performance index. Due to the nonlinear port-Hamiltonian system structure, however, explicit (forward) methods for optimal control of port-Hamiltonian systems require the generally intractable analytical solution of the Hamilton-Jacobi-Bellman equation. Adaptive dynamic programming methods provide a means to circumvent this issue. However, the few existing approaches for port-Hamiltonian systems hinge on very specific sub-classes of either performance indices or system dynamics or require the intransparent guessing of stabilizing initial weights. In this paper, we contribute towards closing this largely unexplored research area by proposing a time-continuous adaptive feedback controller for the optimal control of general time-continuous input-state-output port-Hamiltonian systems with respect to general Lagrangian performance indices. Its control law implements an online learning procedure which uses the Hamiltonian of the system as an initial value function candidate. The time-continuous learning of the value function is achieved by means of a certain Lagrange multiplier that allows to evaluate the optimality of the current solution. In particular, constructive conditions for stabilizing initial weights are stated and asymptotic stability of the closed-loop equilibrium is proven. Our work is concluded by simulations for exemplary linear and nonlinear optimization problems which demonstrate asymptotic convergence of the controllers resulting from the proposed online adaptation procedure.

Full work available at URL: https://arxiv.org/abs/2007.08645

Recommendations

• Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach
• Reinforcement learning for adaptive optimal control of continuous-time linear periodic systems
• Integral \(Q\)-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems
• Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints
• Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

zbMATH Keywords

adaptive controloptimization-based controller synthesisport-Hamiltonian systemsdynamic optimization problem

Mathematics Subject Classification ID

Existence theories for optimal control problems involving partial differential equations (49J20) Feedback control (93B52) Adaptive control/observation systems (93C40) Networked control (93B70)

Cites Work

• A `universal' construction of Artstein's theorem on nonlinear stabilization
• Introduction to topology
• Interconnection and damping assignment passivity-based control: a survey
• Constructive nonlinear control: a historical perspective
• Constructive nonlinear control
• On optimality of passivity based controllers in discrete-time
• Robust adaptive dynamic programming for linear and nonlinear systems: an overview
• Control by Interconnection and Standard Passivity-Based Control of Port-Hamiltonian Systems
• \(L_2\)-gain and passivity techniques in nonlinear control
• Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem
• A port-Hamiltonian approach to power network modeling and analysis
• Robust nonlinear control design. State-space and Lyapunov techniques
• Model-based reinforcement learning for approximate optimal regulation
• Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems
• Reduced order LQG control design for port Hamiltonian systems
• Port-Hamiltonian Systems in Adaptive and Learning Control: A Survey
• A Unifying Energy-Based Approach to Stability of Power Grids With Market Dynamics
• Title not available (Why is that?)
• Algebraic topology. A primer
• Linear algebra

Cited In (7)

• Control of port-Hamiltonian systems with minimal energy supply
• Learning port-Hamiltonian systems -- algorithms
• Learning feedback Nash strategies for nonlinear port-Hamiltonian systems
• An overview on recent machine learning techniques for port Hamiltonian systems
• Optimal Energy Shaping via Neural Approximators
• Cooperative control of NN super twisting sliding mode and EPH methods for uncertain nonlinear systems
• A combined control by interconnection -- model predictive control design for constrained port-Hamiltonian systems

This page was built for publication: Optimal control of port-Hamiltonian systems: a continuous-time learning approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2665352)