Estimation and inference by stochastic optimization

DOI10.1016/J.JECONOM.2023.105638MaRDI QIDQ6193080zbMATH OpenOpenAlexFDO

Authors Jean-Jacques Forneron

Publication date 13 February 2024

Published in Journal of Econometrics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2205.03254

zbMATH Keywords

\(M\) out of \(n\) and multiplier bootstrapM-estimationstochastic gradient descent

Mathematics Subject Classification ID

Statistics (62-XX) Game theory, economics, finance, and other social and behavioral sciences (91-XX)

Abstract: In non-linear estimations, it is common to assess sampling uncertainty by bootstrap inference. For complex models, this can be computationally intensive. This paper combines optimization with resampling: turning stochastic optimization into a fast resampling device. Two methods are introduced: a resampled Newton-Raphson (rNR) and a resampled quasi-Newton (rqN) algorithm. Both produce draws that can be used to compute consistent estimates, confidence intervals, and standard errors in a single run. The draws are generated by a gradient and Hessian (or an approximation) computed from batches of data that are resampled at each iteration. The proposed methods transition quickly from optimization to resampling when the objective is smooth and strictly convex. Simulated and empirical applications illustrate the properties of the methods on large scale and computationally intensive problems. Comparisons with frequentist and Bayesian methods highlight the features of the algorithms.

Cites work

• A bootstrap based on the estimating equations of the linear model
• A fast resample method for parametric and semiparametric models
• A fast subsampling method for nonlinear dynamic models
• A score based approach to wild bootstrap inference
• A Stochastic Approximation Method
• Acceleration of Stochastic Approximation by Averaging
• Adaptive subgradient methods for online learning and stochastic optimization
• An inequality for trace ideals
• Bootstrap Standard Error Estimates for Linear Regression
• Bridging the gap between constant step size stochastic gradient descent and Markov chains
• Convergence of quasi-Newton matrices generated by the symmetric rank one update
• Dynamic discrete choice structural models: a survey
• Estimating Trade Flows: Trading Partners and Trading Volumes^*
• Functions of Matrices
• Higher-Order Improvements of a Computationally Attractive k-Step Bootstrap for Extremum Estimators
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 5060482 (Why is no real title available?)
• scientific article; zbMATH DE number 3307153 (Why is no real title available?)
• Inference from iterative simulation using multiple sequences
• Lectures on convex optimization
• MCMC convergence diagnosis via multivariate bounds on log-concave densities
• Moment consistency of the exchangeably weighted bootstrap for semiparametric M-estimation
• On the computational complexity of MCMC-based estimators in large samples
• Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher
• Poor (wo)man's bootstrap
• Quasi-Newton Methods, Motivation and Theory
• Rates of convergence of the Hastings and Metropolis algorithms
• Some methods of speeding up the convergence of iteration methods
• Split-panel jackknife estimation of fixed-effect models
• Statistical inference for model parameters in stochastic gradient descent
• Stochastic Estimation of the Maximum of a Regression Function
• The convergence of variable metric matrices in unconstrained optimization
• The Stochastic Difference Between Econometric Statistics
• The variation of the spectrum of a normal matrix

Cited in

(1)

• The proximal bootstrap for constrained estimators

This page was built for publication: Estimation and inference by stochastic optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6193080)