A Tutorial on Thompson Sampling
From MaRDI portal
Publication:4556183
Abstract: Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what is known to maximize immediate performance and investing to accumulate new information that may improve future performance. The algorithm addresses a broad range of problems in a computationally efficient manner and is therefore enjoying wide use. This tutorial covers the algorithm and its application, illustrating concepts through a range of examples, including Bernoulli bandit problems, shortest path problems, product recommendation, assortment, active learning with neural networks, and reinforcement learning in Markov decision processes. Most of these problems involve complex information structures, where information revealed by taking an action informs beliefs about other actions. We will also discuss when and why Thompson sampling is or is not effective and relations to alternative algorithms.
Recommendations
- scientific article; zbMATH DE number 7626733
- Linear Thompson sampling revisited
- Thompson sampling: an asymptotically optimal finite-time analysis
- Thompson Sampling for Stochastic Control: The Finite Parameter Case
- Thompson Sampling for Bayesian Bandits with Resets
- Thompson Sampling for Stochastic Control: The Continuous Parameter Case
- Near-optimal regret bounds for Thompson sampling
- An information-theoretic analysis of Thompson sampling
- On the Prior Sensitivity of Thompson Sampling
- Feel-Good Thompson Sampling for Contextual Bandits and Reinforcement Learning
Cited in
(59)- Linear Thompson sampling revisited
- Randomized allocation with nonparametric estimation for contextual multi-armed bandits with delayed rewards
- IntelligentPooling: practical Thompson sampling for mHealth
- Gittins' theorem under uncertainty
- Fundamental design principles for reinforcement learning algorithms
- Choosing the best arm with guaranteed confidence
- On the sample complexity of the linear quadratic regulator
- Ballooning multi-armed bandits
- Quantum greedy algorithms for multi-armed bandits
- Bayesian optimization package: PHYSBO
- An information-theoretic analysis of Thompson sampling
- On the Prior Sensitivity of Thompson Sampling
- Thompson Sampling for Bayesian Bandits with Resets
- A survey on online learning methods: Thompson sampling and others
- scientific article; zbMATH DE number 7370522 (Why is no real title available?)
- scientific article; zbMATH DE number 7626720 (Why is no real title available?)
- scientific article; zbMATH DE number 7626733 (Why is no real title available?)
- SAMBA: A Generic Framework for Secure Federated Multi-Armed Bandits
- Bandit algorithms
- Sliding-Window Thompson Sampling for Non-Stationary Settings
- A contextual bandit bake-off
- Deep exploration via randomized value functions
- Ensemble Kalman sampler: mean-field limit and convergence analysis
- Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits Under Realizability
- Satisficing in Time-Sensitive Bandit Learning
- Fairness improvement for black-box classifiers with Gaussian process
- Online learning of network bottlenecks via minimax paths
- Variable Selection Via Thompson Sampling
- Reward Maximization Through Discrete Active Inference
- Reinforcement Learning, Bit by Bit
- Online learning of energy consumption for navigation of electric vehicles
- Branching time active inference: the theory and its generality
- Thompson sampling for networked control over unknown channels
- Multinomial Thompson sampling for rating scales and prior considerations for calibrating uncertainty
- Nonparametric failure time: time-to-event machine learning with heteroskedastic Bayesian additive regression trees and low information omnibus Dirichlet process mixtures
- Subgroup analysis and adaptive experiments crave for debiasing
- TSEC: A Framework for Online Experimentation under Experimental Constraints
- Thompson sampling-based recursive block elimination for dynamic assignment under limited budget in pure-exploration
- TSPINN: Thompson sampling-based adaptive training for physics-informed neural networks
- A model aggregation approach for high-dimensional large-scale optimization
- Bayesian response-adaptive randomization for cluster randomized controlled trials
- Replicable bandits for digital health interventions
- Counterfactual inference in sequential experiments
- Adaptive maximization of social welfare
- Sequential Monte Carlo bandits
- Contextual bandits with stage-wise constraints
- Adaptive randomization methods for sequential multiple assignment randomized trials (SMARTs) via Thompson sampling
- Continual learning as computationally constrained reinforcement learning
- Adaptive experiments toward learning treatment effect heterogeneity
- Provably efficient information-directed sampling algorithms for multi-agent reinforcement learning
- Policy learning ``without overlap: pessimism and generalized empirical Bernstein's inequality
- Thompson sampling for zero-inflated count outcomes with an application to the drink less mobile health study
- Parallel constrained Bayesian optimization via batched Thompson sampling with enhanced active learning process for reliability-based design optimization
- Sequential Selection for Minimizing the Variance with Application to Crystallization Experiments
- Stochastic Low-Rank Tensor Bandits for Multi-Dimensional Online Decision Making
- Solving Bernoulli rank-one bandits with unimodal Thompson sampling
- Thompson sampling for adversarial bit prediction
- Adaptive network approach to exploration-exploitation trade-off in reinforcement learning
- Explainable parameter calibration via importance-driven sequential design with an application to building energy systems
This page was built for publication: A Tutorial on Thompson Sampling
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4556183)
