Control of chaotic systems by deep reinforcement learning

DOI10.1098/RSPA.2019.0351zbMATH Open1472.68171arXiv1906.07672OpenAlexW3100363258WikidataQ91866606 ScholiaQ91866606MaRDI QIDQ5160799FDOQ5160799

Authors: Michele Alessandro Bucci, Onofrio Semeraro, Alexandre Allauzen, Grzegorz Wiśniewski, L. Mathelin, L. Cordier 

Publication date: 29 October 2021

Published in: Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences (Search for Journal in Brave)

Abstract: Deep Reinforcement Learning (DRL) is applied to control a nonlinear, chaotic system governed by the one-dimensional Kuramoto-Sivashinsky (KS) equation. DRL uses reinforcement learning principles for the determination of optimal control solutions and deep Neural Networks for approximating the value function and the control policy. Recent applications have shown that DRL may achieve superhuman performance in complex cognitive tasks. In this work, we show that using restricted, localized actuations, partial knowledge of the state based on limited sensor measurements, and model-free DRL controllers, it is possible to stabilize the dynamics of the KS system around its unstable fixed solutions, here considered as target states. The robustness of the controllers is tested by considering several trajectories in the phase-space emanating from different initial conditions; we show that the DRL is always capable of driving and stabilizing the dynamics around the target states. The complexity of the KS system, the possibility of defining the DRL control policies by solely relying on the local measurements of the system, and their efficiency in controlling its nonlinear dynamics pave the way for the application of RL methods in control of complex fluid systems such as turbulent boundary layers, turbulent mixers or multiphase flows.

Full work available at URL: https://arxiv.org/abs/1906.07672

Recommendations

• Control of chaos with time-delayed feedback based on deep reinforcement learning
• Optimal chaos control through reinforcement learning
• Repetitive learning control of continuous chaotic systems
• scientific article; zbMATH DE number 2013157
• Control and anti-control of chaos based on the moving largest Lyapunov exponent using reinforcement learning
• Reinforcement learning control of unknown dynamic systems
• scientific article; zbMATH DE number 2230446
• Learning Control of Time-Delay Chaotic Systems and its Applications

zbMATH Keywords

artificial intelligencefluid mechanicsflow controlKuramoto-Sivashinskymechanical engineeringdeep reinforcement learning

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Chaos control for problems involving ordinary differential equations (34H10) Optimality conditions for problems involving ordinary differential equations (49K15) Dynamic programming in optimal control and differential games (49L20) Flow control and optimization for incompressible inviscid fluids (76B75)

Cites Work

• Title not available (Why is that?)
• Deep learning
• \({\mathcal Q}\)-learning
• Title not available (Why is that?)
• Implicit-Explicit Methods for Time-Dependent Partial Differential Equations
• Turbulence, Coherent Structures, Dynamical Systems and Symmetry
• Title not available (Why is that?)
• Optimal control
• Title not available (Why is that?)
• DNS-based predictive control of turbulence: An optimal benchmark for feedback algorithms
• Dynamic programming and stochastic control processes
• Title not available (Why is that?)
• Reinforcement learning. An introduction
• Title not available (Why is that?)
• Title not available (Why is that?)
• Reduced-order models for control of fluids using the eigensystem realization algorithm
• The minimal flow unit in near-wall turbulence
• Title not available (Why is that?)
• A linear systems approach to flow control
• On the state space geometry of the Kuramoto-Sivashinsky flow in a periodic domain
• Feedback control of the Kuramoto-Sivashinsky equation
• The steady states of the Kuramoto-Sivashinsky equation
• Adjoint equations in stability analysis
• Edge states as mediators of bypass transition in boundary-layer flows
• Methods for solution of large optimal control problems that bypass open-loop model reduction
• Dynamical Systems Approach to Turbulence
• Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control
• Nonlinear control of unsteady finite-amplitude perturbations in the Blasius boundary-layer flow

Cited In (21)

• Robust flow control and optimal sensor placement using deep reinforcement learning
• Deep reinforcement learning for the control of conjugate heat transfer
• Reinforcement-learning-based actuator selection method for active flow control
• Explorative gradient method for active drag reduction of the fluidic pinball and slanted Ahmed body
• Comparative analysis of machine learning methods for active flow control
• Dynamic metasurface control using deep reinforcement learning
• A review on deep reinforcement learning for fluid mechanics
• Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution
• Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning
• Control of chaos with time-delayed feedback based on deep reinforcement learning
• Title not available (Why is that?)
• Control strategy of speed servo systems based on deep reinforcement learning
• From the POD-Galerkin method to sparse manifold models
• Optimal chaos control through reinforcement learning
• Reinforcement-learning-based control of convectively unstable flows
• Control and anti-control of chaos based on the moving largest Lyapunov exponent using reinforcement learning
• Near-grazing bifurcations and deep reinforcement learning control of an impact oscillator with elastic constraints
• Proximal policy optimization‐based controller for chaotic systems
• Multi-fidelity reinforcement learning framework for shape optimization
• Feedback control of chaotic systems using multiple shooting shadowing and application to Kuramoto-Sivashinsky equation
• Deep reinforcement learning-based active control for drag reduction of three equilateral-triangular circular cylinders

This page was built for publication: Control of chaotic systems by deep reinforcement learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5160799)