Supervised learning algorithms for controlling underactuated dynamical systems

DOI10.1016/J.PHYSD.2020.132621MaRDI QIDQ2127407zbMATH OpenOpenAlexFDO

Authors Bharat Monga, Jeff Moehlis

Publication date 20 April 2022

Published in Physica D (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1909.11119

zbMATH Keywords

bang bang controlbinary classificationcoupled oscillatorsmachine learningsupervised learningunderactuated dynamical systems

Mathematics Subject Classification ID

Existence of optimal solutions belonging to restricted classes (Lipschitz controls, bang-bang controls, etc.) (49J30) Artificial neural networks and deep learning (68T07) Automated systems (robots, etc.) in control theory (93C85)

Abstract: Control of underactuated dynamical systems has been studied for decades in robotics, and is now emerging in other fields such as neuroscience. Most of the advances have been in model based control theory, which has limitations when the system under study is very complex and it is not possible to construct a model. This calls for data driven control methods like machine learning, which has spread to many fields in the recent years including control theory. However, the success of such algorithms has been dependent on availability of large datasets. Moreover, due to their black box nature, it is challenging to analyze how such algorithms work, which may be crucial in applications where failure is very costly. In this paper, we develop two related novel supervised learning algorithms. The algorithms are powerful enough to control a wide variety of complex underactuated dynamical systems, and yet have a simple and intelligent structure that allows them to work with a sparse data set even in the presence of noise. Our algorithms output a bang-bang (binary) control input by taking in feedback of the state of the dynamical system. The algorithms learn this control input by maximizing a reward function in both short and long time horizons. We demonstrate the versatility of our algorithms by applying them to a diverse range of applications including: switching between bistable states, changing the phase of an oscillator, desynchronizing a population of synchronized coupled oscillators, and stabilizing an unstable fixed point. For most of these applications we are able to reason why our algorithms work by using traditional dynamical systems and control theory. We also compare our learning algorithms with some traditional control algorithms, and reason why our algorithms work better.

Recommendations

• An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems
• scientific article; zbMATH DE number 38962
• Reinforcement learning control of unknown dynamic systems
• On some algorithms of learning control
• Learning optimal control in deterministic systems
• Learning theory and robust control
• Control systems and reinforcement learning
• Regularization and robustness of learning-based control algorithms
• scientific article; zbMATH DE number 817428

Cites work

• A model of desynchronizing deep brain stimulation with a demand-controlled coordinated reset of neural subpopulations
• Bistability in fluctuating environments. Implications in tumor immunology
• Canards for a reduction of the Hodgkin-Huxley equations
• Chemical oscillations, waves, and turbulence
• Deterministic Nonperiodic Flow
• Dynamics and control of a class of underactuated mechanical systems
• Failures of adaptive control theory and their resolution
• From model-based control to data-driven control: survey, classification and perspective
• scientific article; zbMATH DE number 5936982 (Why is no real title available?)
• Introduction to Applied Nonlinear Dynamical Systems and Chaos
• Machine learning: Trends, perspectives, and prospects
• Mathematical physiology
• Minimum energy desynchronizing control for coupled neurons
• Neural networks for control systems - a survey
• On the Phase Reduction and Response Dynamics of Neural Oscillator Populations
• Optimal chaotic desynchronization for neural populations
• Optimal phase control of biological oscillators using augmented phase reduction
• Phase distribution control of a population of oscillators
• Phase reduction and phase-based optimal control for biological systems: a tutorial
• Phase- and Center-Manifold Reductions for Large Populations of Coupled Oscillators with Application to Non-Locally Coupled Systems
• Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
• Starting, stopping, and resetting biological oscillators: in search of optimum perturbations
• The geometry of biological time.
• Time optimal control of spiking neurons

Cited in

(10)

• Leveraging deep learning to control neural oscillators
• Phase distribution control of neural oscillator populations using local radial basis function meshfree technique with application in epileptic seizures: a numerical simulation approach
• Analysis of neural clusters due to deep brain stimulation pulses
• Machine learning control -- taming nonlinear dynamics and turbulence
• scientific article; zbMATH DE number 1560539 (Why is no real title available?)
• An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems
• Machine Learning and Control Theory
• Optimal Control of Oscillation Timing and Entrainment Using Large Magnitude Inputs: An Adaptive Phase-Amplitude-Coordinate-Based Approach
• Data-driven control of oscillator networks with population-level measurement
• Fast optimal entrainment of limit-cycle oscillators by strong periodic inputs via phase-amplitude reduction and Floquet theory

This page was built for publication: Supervised learning algorithms for controlling underactuated dynamical systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2127407)