The unreasonable effectiveness of deep learning in artificial intelligence

DOI10.1073/PNAS.1907373117MaRDI QIDQ5073209zbMATH OpenOpenAlexWikidataFDO

Authors Terrence J. Sejnowski

Publication date 5 May 2022

Published in Proceedings of the National Academy of Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2002.04806

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Abstract: Deep learning networks have been trained to recognize speech, caption photographs and translate text between languages at high levels of performance. Although applications of deep learning networks to real world problems have become ubiquitous, our understanding of why they are so effective is lacking. These empirical results should not be possible according to sample complexity in statistics and non-convex optimization theory. However, paradoxes in the training and effectiveness of deep learning networks are being investigated and insights are being found in the geometry of high-dimensional spaces. A mathematical theory of deep learning would illuminate how they function, allow us to assess the strengths and weaknesses of different network architectures and lead to major improvements. Deep learning has provided natural ways for humans to communicate with digital devices and is foundational for building artificial general intelligence. Deep learning was inspired by the architecture of the cerebral cortex and insights into autonomy and general intelligence may be found in other brain regions that are essential for planning and survival, but major breakthroughs will be needed to achieve these goals.

Recommendations

• The Science of Deep Learning
• Advances in deep learning
• A review of deep learning
• Deep learning
• Deep learning
• A selective overview of deep learning
• Learning deep architectures for AI
• Introduction to deep learning. From logical calculus to artificial intelligence

Cites work

• A Fast Learning Algorithm for Deep Belief Nets
• A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
• A logical calculus of the ideas immanent in nervous activity
• Benign overfitting in linear regression
• scientific article; zbMATH DE number 3314813 (Why is no real title available?)
• Learning representations by back-propagating errors
• Statistical modeling: The two cultures. (With comments and a rejoinder).
• The unreasonable effectiveness of mathematics in the natural sciences. Richard courant lecture in mathematical sciences delivered at New York University, May 11, 1959
• Theoretical issues in deep networks

Cited in

(30)

• Deep blue's contribution to AI
• Why does deep and cheap learning work so well?
• An analytic layer-wise deep learning framework with applications to robotics
• Localized learning: a possible alternative to current deep learning techniques
• A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
• Space, time, categories, mechanics, and consciousness: on Kant and neuroscience
• Obituary: Horace Barlow: a vision scientist for the ages (1922--2020)
• Free dynamics of feature learning processes
• Optimal control by deep learning techniques and its applications on epidemic models
• Neural Networks with Disabilities: An Introduction to Complementary Artificial Intelligence
• Theory of machine learning based on nonrelativistic quantum mechanics
• A mathematical theory of semantic development in deep neural networks
• A contribution to the statistical theory of deep learning
• A formal proof of the expressiveness of deep learning
• A formal proof of the expressiveness of deep learning
• Can We Teach Functions to an Artificial Intelligence by Just Showing It Enough “Ground Truth”?
• Generalized Lyapunov exponents and aspects of the theory of deep learning
• An Unconventional Look at AI: Why Today’s Machine Learning Systems are not Intelligent
• Understanding the role of pathways in a deep neural network
• Discovering first principle of behavioural change in disease transmission dynamics by deep learning
• Hyper-flexible convolutional neural networks based on generalized Lehmer and power means
• Compositional sparsity of learnable functions
• Explaining answers generated by knowledge graph embeddings
• Elementary proof of Funahashi's theorem
• Estimating time-varying reproduction number by deep learning techniques
• Differential equations for continuous-time deep learning
• The discrete inverse conductivity problem solved by the weights of an interpretable neural network
• Tensor decomposition-based neural operator with dynamic mode decomposition for parameterized time-dependent problems
• Phase transition analysis for shallow neural networks with arbitrary activation functions
• Editorial introduction to the neural networks special issue on deep learning of representations

This page was built for publication: The unreasonable effectiveness of deep learning in artificial intelligence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5073209)