On universal prediction and Bayesian confirmation

DOI10.1016/J.TCS.2007.05.016MaRDI QIDQ2382281zbMATH OpenOpenAlexWikidataFDO

Authors Marcus Hutter

Publication date 28 September 2007

Published in Theoretical Computer Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0709.1516

zbMATH Keywords

Bayesblack raven paradoxconfirmation theoryKolmogorov complexitymodel classesnoncomputable environmentsOccam's razorold-evidence/updating problemphilosophical issuesprediction boundsreparametrization invarianceSequence predictionSolomonoff priorsymmetry principle

Mathematics Subject Classification ID

Algorithmic information theory (Kolmogorov complexity, etc.) (68Q30) Computational learning theory (68Q32)

Abstract: The Bayesian framework is a well-studied and successful framework for inductive reasoning, which includes hypothesis testing and confirmation, parameter estimation, sequence prediction, classification, and regression. But standard statistical guidelines for choosing the model class and prior are not always available or fail, in particular in complex situations. Solomonoff completed the Bayesian framework by providing a rigorous, unique, formal, and universal choice for the model class and the prior. We discuss in breadth how and in which sense universal (non-i.i.d.) sequence prediction solves various (philosophical) problems of traditional Bayesian sequence prediction. We show that Solomonoff's model possesses many desirable properties: Strong total and weak instantaneous bounds, and in contrast to most classical continuous prior densities has no zero p(oste)rior problem, i.e. can confirm universal hypotheses, is reparametrization and regrouping invariant, and avoids the old-evidence and updating problem. It even performs well (actually better) in non-computable environments.

Recommendations

• Theory and Applications of Models of Computation
• 10.1162/1532443041827952
• Universal prediction of selected bits
• Universality of Bayesian mixture predictors
• Convergence and Error Bounds for Universal Prediction of Nonbinary Sequences

Cites work

• 10.1162/1532443041827952
• A formal theory of inductive inference. Part II
• A Theory of Program Size Formally Identical to Information Theory
• A universal prior for integers and estimation by minimum description length
• Algorithmic complexity bounds on future prediction errors
• Algorithmic Learning Theory
• Algorithmic Learning Theory
• Algorithmic Learning Theory
• An invariant form for the prior probability in estimation problems
• Clustering by Compression
• Complexity-based induction systems: Comparisons and convergence theorems
• Convergence and loss bounds for bayesian sequence prediction
• HIERARCHIES OF GENERALIZED KOLMOGOROV COMPLEXITIES AND NONENUMERABLE UNIVERSAL MEASURES COMPUTABLE IN THE LIMIT
• scientific article; zbMATH DE number 2089364 (Why is no real title available?)
• scientific article; zbMATH DE number 3427210 (Why is no real title available?)
• scientific article; zbMATH DE number 4211134 (Why is no real title available?)
• scientific article; zbMATH DE number 3667770 (Why is no real title available?)
• scientific article; zbMATH DE number 45100 (Why is no real title available?)
• scientific article; zbMATH DE number 3492569 (Why is no real title available?)
• scientific article; zbMATH DE number 1010621 (Why is no real title available?)
• scientific article; zbMATH DE number 845703 (Why is no real title available?)
• scientific article; zbMATH DE number 1391397 (Why is no real title available?)
• scientific article; zbMATH DE number 2188025 (Why is no real title available?)
• scientific article; zbMATH DE number 3189754 (Why is no real title available?)
• scientific article; zbMATH DE number 3074488 (Why is no real title available?)
• scientific article; zbMATH DE number 3074489 (Why is no real title available?)
• Information-theoretic asymptotics of Bayes methods
• Laplace's law of succession and universal encoding
• New error bounds for Solomonoff prediction
• On generalized computable universal priors and their convergence
• On the Complexity of Finite Sequences
• On the existence and convergence of computable universal priors
• On the relation between descriptional complexity and algorithmic probability
• Optimal ordered problem solver
• Present Position and Potential Developments: Some Personal Views: Statistical Theory: The Prequential Approach
• THE COMPLEXITY OF FINITE OBJECTS AND THE DEVELOPMENT OF THE CONCEPTS OF INFORMATION AND RANDOMNESS BY MEANS OF THE THEORY OF ALGORITHMS
• The Selection of Prior Distributions by Formal Rules
• Theory and Applications of Models of Computation
• Universal prediction

Cited in

(22)

• Open problems in universal induction \& intelligence
• A complete theory of everything (will be subjective)
• A generalized characterization of algorithmic probability
• Putnam's diagonal argument and the impossibility of a universal learning machine
• Absolutely no free lunches!
• On Martin-Löf (non-)convergence of Solomonoff's universal mixture
• Universality of Bayesian predictions
• Rejoinder: ``Universality of Bayesian predictions
• Solomonoff induction violates Nicod's criterion
• Concentration and confidence for discrete Bayesian sequence predictors
• Universal prediction of selected bits
• A philosophical treatise of universal induction
• Universal prediction
• Mechanical learners pay a price for Bayesianism
• On Martin-Löf Convergence of Solomonoff’s Mixture
• Theory and Applications of Models of Computation
• Simplicity bias, algorithmic probability, and the random logistic map
• A dilemma for Solomonoff prediction
• Solomonoff prediction and Occam's razor
• Curious coincidences and Kolmogorov complexity
• Hydrozip: how hydrological knowledge can be used to improve compression of hydrological data
• Practical robust estimators for the imprecise Dirichlet model

This page was built for publication: On universal prediction and Bayesian confirmation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2382281)