Control of chaotic systems by deep reinforcement learning
From MaRDI portal
Publication:5160799
artificial intelligencefluid mechanicsflow controlKuramoto-Sivashinskymechanical engineeringdeep reinforcement learning
Artificial neural networks and deep learning (68T07) Chaos control for problems involving ordinary differential equations (34H10) Optimality conditions for problems involving ordinary differential equations (49K15) Dynamic programming in optimal control and differential games (49L20) Flow control and optimization for incompressible inviscid fluids (76B75)
Abstract: Deep Reinforcement Learning (DRL) is applied to control a nonlinear, chaotic system governed by the one-dimensional Kuramoto-Sivashinsky (KS) equation. DRL uses reinforcement learning principles for the determination of optimal control solutions and deep Neural Networks for approximating the value function and the control policy. Recent applications have shown that DRL may achieve superhuman performance in complex cognitive tasks. In this work, we show that using restricted, localized actuations, partial knowledge of the state based on limited sensor measurements, and model-free DRL controllers, it is possible to stabilize the dynamics of the KS system around its unstable fixed solutions, here considered as target states. The robustness of the controllers is tested by considering several trajectories in the phase-space emanating from different initial conditions; we show that the DRL is always capable of driving and stabilizing the dynamics around the target states. The complexity of the KS system, the possibility of defining the DRL control policies by solely relying on the local measurements of the system, and their efficiency in controlling its nonlinear dynamics pave the way for the application of RL methods in control of complex fluid systems such as turbulent boundary layers, turbulent mixers or multiphase flows.
Recommendations
- Control of chaos with time-delayed feedback based on deep reinforcement learning
- Optimal chaos control through reinforcement learning
- Repetitive learning control of continuous chaotic systems
- scientific article; zbMATH DE number 2013157
- Control and anti-control of chaos based on the moving largest Lyapunov exponent using reinforcement learning
- Reinforcement learning control of unknown dynamic systems
- scientific article; zbMATH DE number 2230446
- Learning Control of Time-Delay Chaotic Systems and its Applications
Cites work
- scientific article; zbMATH DE number 3128787 (Why is no real title available?)
- scientific article; zbMATH DE number 3168214 (Why is no real title available?)
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 1090982 (Why is no real title available?)
- scientific article; zbMATH DE number 3997615 (Why is no real title available?)
- scientific article; zbMATH DE number 804845 (Why is no real title available?)
- scientific article; zbMATH DE number 870530 (Why is no real title available?)
- scientific article; zbMATH DE number 2238822 (Why is no real title available?)
- A linear systems approach to flow control
- Adjoint equations in stability analysis
- Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control
- DNS-based predictive control of turbulence: An optimal benchmark for feedback algorithms
- Deep learning
- Dynamic programming and stochastic control processes
- Dynamical Systems Approach to Turbulence
- Edge states as mediators of bypass transition in boundary-layer flows
- Feedback control of the Kuramoto-Sivashinsky equation
- Implicit-Explicit Methods for Time-Dependent Partial Differential Equations
- Methods for solution of large optimal control problems that bypass open-loop model reduction
- Nonlinear control of unsteady finite-amplitude perturbations in the Blasius boundary-layer flow
- On the state space geometry of the Kuramoto-Sivashinsky flow in a periodic domain
- Optimal control
- Reduced-order models for control of fluids using the eigensystem realization algorithm
- Reinforcement learning. An introduction
- The minimal flow unit in near-wall turbulence
- The steady states of the Kuramoto-Sivashinsky equation
- Turbulence, Coherent Structures, Dynamical Systems and Symmetry
- \({\mathcal Q}\)-learning
Cited in
(21)- A review on deep reinforcement learning for fluid mechanics
- Comparative analysis of machine learning methods for active flow control
- Dynamic metasurface control using deep reinforcement learning
- Control of chaos with time-delayed feedback based on deep reinforcement learning
- Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution
- Reinforcement-learning-based actuator selection method for active flow control
- Near-grazing bifurcations and deep reinforcement learning control of an impact oscillator with elastic constraints
- Proximal policy optimization‐based controller for chaotic systems
- Deep reinforcement learning-based active control for drag reduction of three equilateral-triangular circular cylinders
- Control and anti-control of chaos based on the moving largest Lyapunov exponent using reinforcement learning
- Optimal chaos control through reinforcement learning
- Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning
- Multi-fidelity reinforcement learning framework for shape optimization
- Robust flow control and optimal sensor placement using deep reinforcement learning
- Deep reinforcement learning for the control of conjugate heat transfer
- From the POD-Galerkin method to sparse manifold models
- scientific article; zbMATH DE number 2013157 (Why is no real title available?)
- Reinforcement-learning-based control of convectively unstable flows
- Explorative gradient method for active drag reduction of the fluidic pinball and slanted Ahmed body
- Feedback control of chaotic systems using multiple shooting shadowing and application to Kuramoto-Sivashinsky equation
- Control strategy of speed servo systems based on deep reinforcement learning
This page was built for publication: Control of chaotic systems by deep reinforcement learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5160799)
