Superintelligence cannot be contained: lessons from computability theory

DOI10.1613/JAIR.1.12202MaRDI QIDQ5145835zbMATH OpenOpenAlexWikidataFDO

Authors Manuel Alfonseca, Manuel Cebrian, Antonio Fernández Anta, Lorenzo Coviello, Andrés Abeliuk, Iyad Rahwan

Publication date 22 January 2021

Published in Journal of Artificial Intelligence Research (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1607.00913

zbMATH Keywords

computational social systemsmathematical foundationsphilosophical foundations

Mathematics Subject Classification ID

Classical models of computation (Turing machines, etc.) (68Q04) General topics in artificial intelligence (68T01)

Abstract: Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potential catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible.

Recommendations

• Hypercomputation: fantasy or reality? A position paper
• Hypercomputation: Philosophical issues
• Singularity hypotheses. A scientific and philosophical assessment
• Strong determinism vs. computability
• The concept of computability

Cites work

• A probabilistic anytime algorithm for the halting problem
• An Unsolvable Problem of Elementary Number Theory
• Checkers is solved
• Classes of Recursively Enumerable Sets and Their Decision Problems
• Deep Blue
• Fundamentals of Computation Theory
• scientific article; zbMATH DE number 783783 (Why is no real title available?)
• On Computable Numbers, with an Application to the Entscheidungsproblem
• On non-computable functions

Cited in

(1)

• Exploring the constraints on artificial general intelligence: a game-theoretic model of human vs machine interaction

This page was built for publication: Superintelligence cannot be contained: lessons from computability theory

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5145835)