Inverse Bayesian optimization: learning human acquisition functions in an exploration vs exploitation search task

DOI10.1214/21-BA1303MaRDI QIDQ6122013zbMATH OpenOpenAlexFDO

Authors Nathan Sandholtz, Yohsuke Miyamoto, Luke Bornn, Maurice A. Smith

Publication date 27 February 2024

Published in Bayesian Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2104.09237, https://projecteuclid.org/journals/bayesian-analysis/volume-18/issue-1/Inverse-Bayesian-Optimization-Learning-Human-Acquisition-Functions-in-an/10.1214/21-BA1303.full

zbMATH Keywords

Bayesian optimizationdirectional statisticsexploration vs. exploitationhuman cognitioninverse optimizationlab experimentprobabilistic models

Mathematics Subject Classification ID

Statistics (62-XX)

Abstract: This paper introduces a probabilistic framework to estimate parameters of an acquisition function given observed human behavior that can be modeled as a collection of sample paths from a Bayesian optimization procedure. The methodology involves defining a likelihood on observed human behavior from an optimization task, where the likelihood is parameterized by a Bayesian optimization subroutine governed by an unknown acquisition function. This structure enables us to make inference on a subject's acquisition function while allowing their behavior to deviate around the solution to the Bayesian optimization subroutine. To test our methods, we designed a sequential optimization task which forced subjects to balance exploration and exploitation in search of an invisible target location. Applying our proposed methods to the resulting data, we find that many subjects tend to exhibit exploration preferences beyond that of standard acquisition functions to capture. Guided by the model discrepancies, we augment the candidate acquisition functions to yield a superior fit to the human behavior in this task.

Cites work

• Data-driven inverse optimization with imperfect information
• Efficient global optimization of expensive black-box functions
• scientific article; zbMATH DE number 3567782 (Why is no real title available?)
• scientific article; zbMATH DE number 3612796 (Why is no real title available?)
• Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
• Inverse Decision Theory
• Inverse Optimization
• Inverse optimization with noisy data
• Prospect Theory: An Analysis of Decision under Risk
• Robust inverse optimization

Cited in

(1)

• Learning risk preferences in Markov decision processes: an application to the fourth down decision in the national football league

This page was built for publication: Inverse Bayesian optimization: learning human acquisition functions in an exploration vs exploitation search task

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6122013)