Supervised learning algorithms for controlling underactuated dynamical systems

DOI10.1016/J.PHYSD.2020.132621zbMATH Open1489.93083arXiv1909.11119OpenAlexW3036566854MaRDI QIDQ2127407FDOQ2127407

Authors: Bharat Monga, Jeff Moehlis 

Publication date: 20 April 2022

Published in: Physica D (Search for Journal in Brave)

Abstract: Control of underactuated dynamical systems has been studied for decades in robotics, and is now emerging in other fields such as neuroscience. Most of the advances have been in model based control theory, which has limitations when the system under study is very complex and it is not possible to construct a model. This calls for data driven control methods like machine learning, which has spread to many fields in the recent years including control theory. However, the success of such algorithms has been dependent on availability of large datasets. Moreover, due to their black box nature, it is challenging to analyze how such algorithms work, which may be crucial in applications where failure is very costly. In this paper, we develop two related novel supervised learning algorithms. The algorithms are powerful enough to control a wide variety of complex underactuated dynamical systems, and yet have a simple and intelligent structure that allows them to work with a sparse data set even in the presence of noise. Our algorithms output a bang-bang (binary) control input by taking in feedback of the state of the dynamical system. The algorithms learn this control input by maximizing a reward function in both short and long time horizons. We demonstrate the versatility of our algorithms by applying them to a diverse range of applications including: switching between bistable states, changing the phase of an oscillator, desynchronizing a population of synchronized coupled oscillators, and stabilizing an unstable fixed point. For most of these applications we are able to reason why our algorithms work by using traditional dynamical systems and control theory. We also compare our learning algorithms with some traditional control algorithms, and reason why our algorithms work better.

Full work available at URL: https://arxiv.org/abs/1909.11119

Recommendations

• An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems
• scientific article; zbMATH DE number 38962
• Reinforcement learning control of unknown dynamic systems
• On some algorithms of learning control
• Learning optimal control in deterministic systems
• Learning theory and robust control
• Control systems and reinforcement learning
• Regularization and robustness of learning-based control algorithms
• scientific article; zbMATH DE number 817428

zbMATH Keywords

supervised learningmachine learningbinary classificationcoupled oscillatorsbang bang controlunderactuated dynamical systems

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Existence of optimal solutions belonging to restricted classes (Lipschitz controls, bang-bang controls, etc.) (49J30) Automated systems (robots, etc.) in control theory (93C85)

Cites Work

• Machine learning: Trends, perspectives, and prospects
• Deterministic Nonperiodic Flow
• Mathematical physiology
• On the Phase Reduction and Response Dynamics of Neural Oscillator Populations
• The geometry of biological time.
• Chemical oscillations, waves, and turbulence
• Neural networks for control systems - a survey
• Introduction to Applied Nonlinear Dynamical Systems and Chaos
• Canards for a reduction of the Hodgkin-Huxley equations
• Title not available (Why is that?)
• Failures of adaptive control theory and their resolution
• A model of desynchronizing deep brain stimulation with a demand-controlled coordinated reset of neural subpopulations
• Starting, stopping, and resetting biological oscillators: in search of optimum perturbations
• Bistability in fluctuating environments. Implications in tumor immunology
• Dynamics and control of a class of underactuated mechanical systems
• Minimum energy desynchronizing control for coupled neurons
• Time optimal control of spiking neurons
• From model-based control to data-driven control: survey, classification and perspective
• Phase reduction and phase-based optimal control for biological systems: a tutorial
• Phase distribution control of a population of oscillators
• Optimal chaotic desynchronization for neural populations
• Phase- and Center-Manifold Reductions for Large Populations of Coupled Oscillators with Application to Non-Locally Coupled Systems
• Optimal phase control of biological oscillators using augmented phase reduction
• Sparse identification of nonlinear dynamics for model predictive control in the low-data limit

Cited In (10)

• Data-driven control of oscillator networks with population-level measurement
• Fast optimal entrainment of limit-cycle oscillators by strong periodic inputs via phase-amplitude reduction and Floquet theory
• Optimal Control of Oscillation Timing and Entrainment Using Large Magnitude Inputs: An Adaptive Phase-Amplitude-Coordinate-Based Approach
• Analysis of neural clusters due to deep brain stimulation pulses
• Machine Learning and Control Theory
• Machine learning control -- taming nonlinear dynamics and turbulence
• An Algorithm for Learning Without External Supervision and Its Application to Learning Control Systems
• Title not available (Why is that?)
• Leveraging deep learning to control neural oscillators
• Phase distribution control of neural oscillator populations using local radial basis function meshfree technique with application in epileptic seizures: a numerical simulation approach

This page was built for publication: Supervised learning algorithms for controlling underactuated dynamical systems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2127407)