Control of chaotic systems by deep reinforcement learning
From MaRDI portal
artificial intelligencedeep reinforcement learningflow controlfluid mechanicsKuramoto-Sivashinskymechanical engineering
Chaos control for problems involving ordinary differential equations (34H10) Optimality conditions for problems involving ordinary differential equations (49K15) Dynamic programming in optimal control and differential games (49L20) Artificial neural networks and deep learning (68T07) Flow control and optimization for incompressible inviscid fluids (76B75)
Abstract: Deep Reinforcement Learning (DRL) is applied to control a nonlinear, chaotic system governed by the one-dimensional Kuramoto-Sivashinsky (KS) equation. DRL uses reinforcement learning principles for the determination of optimal control solutions and deep Neural Networks for approximating the value function and the control policy. Recent applications have shown that DRL may achieve superhuman performance in complex cognitive tasks. In this work, we show that using restricted, localized actuations, partial knowledge of the state based on limited sensor measurements, and model-free DRL controllers, it is possible to stabilize the dynamics of the KS system around its unstable fixed solutions, here considered as target states. The robustness of the controllers is tested by considering several trajectories in the phase-space emanating from different initial conditions; we show that the DRL is always capable of driving and stabilizing the dynamics around the target states. The complexity of the KS system, the possibility of defining the DRL control policies by solely relying on the local measurements of the system, and their efficiency in controlling its nonlinear dynamics pave the way for the application of RL methods in control of complex fluid systems such as turbulent boundary layers, turbulent mixers or multiphase flows.
Recommendations
- Control of chaos with time-delayed feedback based on deep reinforcement learning
- Optimal chaos control through reinforcement learning
- Repetitive learning control of continuous chaotic systems
- scientific article; zbMATH DE number 2013157
- Control and anti-control of chaos based on the moving largest Lyapunov exponent using reinforcement learning
- Reinforcement learning control of unknown dynamic systems
- scientific article; zbMATH DE number 2230446
- Learning Control of Time-Delay Chaotic Systems and its Applications
Cites work
- \({\mathcal Q}\)-learning
- A linear systems approach to flow control
- Adjoint equations in stability analysis
- Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control
- Deep learning
- DNS-based predictive control of turbulence: An optimal benchmark for feedback algorithms
- Dynamic programming and stochastic control processes
- Dynamical Systems Approach to Turbulence
- Edge states as mediators of bypass transition in boundary-layer flows
- Feedback control of the Kuramoto-Sivashinsky equation
- scientific article; zbMATH DE number 3128787 (Why is no real title available?)
- scientific article; zbMATH DE number 3168214 (Why is no real title available?)
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 1090982 (Why is no real title available?)
- scientific article; zbMATH DE number 3997615 (Why is no real title available?)
- scientific article; zbMATH DE number 804845 (Why is no real title available?)
- scientific article; zbMATH DE number 870530 (Why is no real title available?)
- scientific article; zbMATH DE number 2238822 (Why is no real title available?)
- Implicit-Explicit Methods for Time-Dependent Partial Differential Equations
- Methods for solution of large optimal control problems that bypass open-loop model reduction
- Nonlinear control of unsteady finite-amplitude perturbations in the Blasius boundary-layer flow
- On the state space geometry of the Kuramoto-Sivashinsky flow in a periodic domain
- Optimal control
- Reduced-order models for control of fluids using the eigensystem realization algorithm
- Reinforcement learning. An introduction
- The minimal flow unit in near-wall turbulence
- The steady states of the Kuramoto-Sivashinsky equation
- Turbulence, Coherent Structures, Dynamical Systems and Symmetry
Cited in
(27)- Control and anti-control of chaos based on the moving largest Lyapunov exponent using reinforcement learning
- Deep reinforcement learning for the control of conjugate heat transfer
- Dynamic metasurface control using deep reinforcement learning
- A review on deep reinforcement learning for fluid mechanics
- Control strategy of speed servo systems based on deep reinforcement learning
- Multi-fidelity reinforcement learning framework for shape optimization
- scientific article; zbMATH DE number 2013157 (Why is no real title available?)
- Optimal chaos control through reinforcement learning
- From the POD-Galerkin method to sparse manifold models
- Explorative gradient method for active drag reduction of the fluidic pinball and slanted Ahmed body
- Feedback control of chaotic systems using multiple shooting shadowing and application to Kuramoto-Sivashinsky equation
- Robust flow control and optimal sensor placement using deep reinforcement learning
- Reinforcement-learning-based control of convectively unstable flows
- Reinforcement-learning-based actuator selection method for active flow control
- Comparative analysis of machine learning methods for active flow control
- Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution
- Near-grazing bifurcations and deep reinforcement learning control of an impact oscillator with elastic constraints
- Proximal policy optimization‐based controller for chaotic systems
- Control of chaos with time-delayed feedback based on deep reinforcement learning
- Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning
- Deep reinforcement learning-based active control for drag reduction of three equilateral-triangular circular cylinders
- Harnessing machine learning for identifying parameters in fractional chaotic systems
- Reinforcement-learning-assisted control of four-roll Mills: geometric symmetry and inertial effect
- Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
- Physics-informed dyna-style model-based deep reinforcement learning for dynamic control
- Taming chimeras in coupled oscillators using soft actor-critic based reinforcement learning
- OGF: an online gradient flow method for optimizing the statistical steady-state time averages of unsteady turbulent flows
This page was built for publication: Control of chaotic systems by deep reinforcement learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5160799)
