Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning

DOI10.1016/J.PHYSD.2024.134096MaRDI QIDQ6424398zbMATH OpenOpenAlexFDO

Authors Sebastian Peitz, Jan Stenner, Vikas Chidananda, Oliver Wallscheid, S. L. Brunton, Kunihiko Taira

Publication date 25 January 2023

Full work available at URL https://doi.org/10.1016/j.physd.2024.134096

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Multi-agent systems (93A16) Control/observation systems governed by partial differential equations (93C20)

Abstract: We present a convolutional framework which significantly reduces the complexity and thus, the computational effort for distributed reinforcement learning control of dynamical systems governed by partial differential equations (PDEs). Exploiting translational invariances, the high-dimensional distributed control problem can be transformed into a multi-agent control problem with many identical, uncoupled agents. Furthermore, using the fact that information is transported with finite velocity in many cases, the dimension of the agents' environment can be drastically reduced using a convolution operation over the state space of the PDE. In this setting, the complexity can be flexibly adjusted via the kernel width or by using a stride greater than one. Moreover, scaling from smaller to larger systems -- or the transfer between different domains -- becomes a straightforward task requiring little effort. We demonstrate the performance of the proposed framework using several PDE examples with increasing complexity, where stabilization is achieved by training a low-dimensional deep deterministic policy gradient agent using minimal computing resources.

Recommendations

• Online identification and control of PDEs via reinforcement learning methods
• Neural Control of Parametric Solutions for High-Dimensional Evolution PDEs
• Control of chaotic systems by deep reinforcement learning
• Model-free distributed optimal control for general discrete-time linear systems using reinforcement learning
• Control of partial differential equations via physics-informed neural networks

Cited in

(4)

• Online identification and control of PDEs via reinforcement learning methods
• Transformer-based in-context policy learning for efficient active flow control across various airfoils
• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• Equivariance and partial observations in Koopman operator theory for partial differential equations

This page was built for publication: Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6424398)