Overcoming catastrophic forgetting in neural networks

DOI10.1073/pnas.1611835114zbMath1404.92015arXiv1612.00796OpenAlexW2560647685WikidataQ37737121 ScholiaQ37737121MaRDI QIDQ4646167

John Quan, Kieran Milan, Claudia Clopath, Andrei A. Rusu, Neil Rabinowitz, Raia Hadsell, Razvan Pascanu, Tiago Ramalho, Dharshan Kumaran, Demis Hassabis, Guillaume Desjardins, Agnieszka Grabska-Barwińska, James Kirkpatrick, Joel Veness

Publication date: 11 January 2019

Published in: Proceedings of the National Academy of Sciences (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1612.00796

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20)

Related Items (34)

Stimulus-Driven and Spontaneous Dynamics in Excitatory-Inhibitory Recurrent Neural Networks for Sequence Representation ⋮ Sequential changepoint detection in neural networks with checkpoints ⋮ An analytical theory of curriculum learning in teacher–student networks* ⋮ The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima ⋮ Drifting neuronal representations: bug or feature? ⋮ Single Circuit in V1 Capable of Switching Contexts During Movement Using an Inhibitory Population as a Switch ⋮ Model-Centric Data Manifold: The Data Through the Eyes of the Model ⋮ Blessing of dimensionality at the edge and geometry of few-shot learning ⋮ Deep Bayesian unsupervised lifelong learning ⋮ Continuous learning of spiking networks trained with local rules ⋮ Accelerating algebraic multigrid methods via artificial neural networks ⋮ Adaptive learning of effective dynamics for online modeling of complex systems ⋮ Robust federated learning under statistical heterogeneity via hessian-weighted aggregation ⋮ Hierarchically structured task-agnostic continual learning ⋮ Reliable extrapolation of deep neural operators informed by physics or sparse observations ⋮ KS(conf): a light-weight test if a multiclass classifier operates outside of its specifications ⋮ Automated Deep Learning: Neural Architecture Search Is Not the End ⋮ Accelerating actor-critic-based algorithms via pseudo-labels derived from prior knowledge ⋮ Quantum continual learning of quantum data realizing knowledge backward transfer ⋮ A neural model of schemas and memory encoding ⋮ Toward Training Recurrent Neural Networks for Lifelong Learning ⋮ Deep Reinforcement Learning: A State-of-the-Art Walkthrough ⋮ Progressive learning: a deep learning framework for continual learning ⋮ Learning deep optimizer for blind image deconvolution ⋮ A Minimum Free Energy Model of Motor Learning ⋮ One Step Back, Two Steps Forward: Interference and Learning in Recurrent Neural Networks ⋮ Adversarial Feature Alignment: Avoid Catastrophic Forgetting in Incremental Task Lifelong Learning ⋮ Bayesian Filtering with Multiple Internal Models: Toward a Theory of Social Intelligence ⋮ Learning Invariant Features in Modulatory Networks through Conflict and Ambiguity ⋮ A neurodynamic model of the interaction between color perception and color memory ⋮ Reinforcement Learning in Sparse-Reward Environments With Hindsight Policy Gradients ⋮ Universal statistics of Fisher information in deep neural networks: mean field approach^* ⋮ Unnamed Item ⋮ Adaptive infinite dropout for noisy and sparse data streams

This page was built for publication: Overcoming catastrophic forgetting in neural networks