Bias reduction in sample-based optimization
From MaRDI portal
Abstract: We consider stochastic optimization problems which use observed data to estimate essential characteristics of the random quantities involved. Sample average approximation (SAA) or empirical (plug-in) estimation are very popular ways to use data in optimization. It is well known that sample average optimization suffers from downward bias. We propose to use smooth estimators rather than empirical ones in optimization problems. We establish consistency results for the optimal value and the set of optimal solutions of the new problem formulation. The performance of the proposed approach is compared to SAA theoretically and numerically. We analyze the bias of the new problems and identify sufficient conditions for ensuring less biased estimation of the optimal value of the true problem. At the same time, the error of the new estimator remains controlled. We show that those conditions are satisfied for many popular statistical problems such as regression models, classification problems, and optimization problems with Average (Conditional) Value-at-Risk. We have observed that smoothing the least-squares objective in a regression problem by a normal kernel leads to a ridge regression. Our numerical experience shows that the new estimators frequently exhibit also smaller variance and smaller mean-square error than those of SAA.
Recommendations
- Robust sample average approximation
- Bias Reduction using Stochastic Approximation
- A probability metrics approach for reducing the bias of optimality gap estimators in two-stage stochastic linear programming
- The impact of sampling methods on bias and variance in stochastic linear programs
- When can we improve on sample average approximation for stochastic optimization?
Cites work
- scientific article; zbMATH DE number 124611 (Why is no real title available?)
- scientific article; zbMATH DE number 3635825 (Why is no real title available?)
- scientific article; zbMATH DE number 2121076 (Why is no real title available?)
- A branch and bound method for stochastic global optimization
- A sequential sampling procedure for stochastic programming
- Analysis of Sample-Path Optimization
- Approximations to stochastic programs with complete fixed recourse
- Assessing solution quality in stochastic programs
- Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems
- Consistency of the kernel density estimator: a survey
- Convergence of the empirical mean method in statistics and stochastic programming
- Density estimation
- Differential stability of two-stage stochastic programs
- Epi‐consistency of convex stochastic programs
- From empirical observations to tree models for stochastic optimization: convergence properties
- Introduction to nonparametric estimation
- Lectures on stochastic programming. Modeling and theory.
- Mathematical foundations of infinite-dimensional statistical models
- Mitigating Uncertainty via Compromise Decisions in Two-Stage Stochastic Linear Programming: Variance Reduction
- Monte Carlo bounding techniques for determinig solution quality in stochastic programs
- Necessary and sufficient conditions for weak convergence of smoothed empirical processes.
- Non-asymptotic confidence bounds for the optimal value of a stochastic program
- On the Glivenko-Cantelli problem in stochastic programming: Linear recourse and extensions
- Randomized smoothing for stochastic optimization
- Sample average approximation method for compound stochastic optimization problems
- Sample-path solution of stochastic variational inequalities
- Sampling-based decomposition methods for multistage stochastic programs based on extended polyhedral risk measures
- Stability of Multistage Stochastic Programs
- Statistical estimation of composite risk functionals and risk optimization problems
- Statistical inference of stochastic optimization problems
- Strong convergence of estimators as _n-minimisers of optimisation problems
- The Minimization of Semicontinuous Functions: Mollifier Subgradients
- The sample average approximation method for stochastic discrete optimization
- Uniform central limit theorems for kernel density estimators
- Uniform in bandwidth consistency of kernel-type function estimators
- Weak and strong uniform consistency of the kernel estimate of a density and its derivatives
- Weighted uniform consistency of kernel density estimators.
Cited in
(6)- Debiasing in-sample policy performance for small-data, large-scale optimization
- scientific article; zbMATH DE number 2045445 (Why is no real title available?)
- Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective
- scientific article; zbMATH DE number 4106250 (Why is no real title available?)
- A probability metrics approach for reducing the bias of optimality gap estimators in two-stage stochastic linear programming
- Central limit theorems for vector-valued composite functionals with smoothing and applications
This page was built for publication: Bias reduction in sample-based optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5026842)
