Adaptive System Optimization Using Random Directions Stochastic Approximation

DOI10.1109/TAC.2016.2600643MaRDI QIDQ5280408zbMATH OpenOpenAlexFDO

Authors L. A. Prashanth, Shalabh Bhatnagar, Michael C. Fu, Steve Marcus

Publication date 27 July 2017

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1502.05577

Mathematics Subject Classification ID

Stochastic programming (90C15) Nonconvex programming, global optimization (90C26) Derivative-free methods and methods using generalized derivatives (90C56) Adaptive control/observation systems (93C40)

Abstract: We present novel algorithms for simulation optimization using random directions stochastic approximation (RDSA). These include first-order (gradient) as well as second-order (Newton) schemes. We incorporate both continuous-valued as well as discrete-valued perturbations into both our algorithms. The former are chosen to be independent and identically distributed (i.i.d.) symmetric, uniformly distributed random variables (r.v.), while the latter are i.i.d., asymmetric, Bernoulli r.v.s. Our Newton algorithm, with a novel Hessian estimation scheme, requires N-dimensional perturbations and three loss measurements per iteration, whereas the simultaneous perturbation Newton search algorithm of [1] requires 2N-dimensional perturbations and four loss measurements per iteration. We prove the unbiasedness of both gradient and Hessian estimates and asymptotic (strong) convergence for both first-order and second-order schemes. We also provide asymptotic normality results, which in particular establish that the asymmetric Bernoulli variant of Newton RDSA method is better than 2SPSA of [1]. Numerical experiments are used to validate the theoretical results.

Recommendations

• Adaptive stochastic optimization techniques with applications
• scientific article; zbMATH DE number 825467
• scientific article; zbMATH DE number 3963705
• scientific article; zbMATH DE number 703401
• scientific article; zbMATH DE number 53271
• scientific article; zbMATH DE number 3908280
• Adaptive stochastic approximation algorithm
• Adaptive approximation models in optimization
• scientific article; zbMATH DE number 1569106
• Adaptive Sequential Stochastic Optimization

Cited in

(6)

• On stochastic extremum seeking via adaptive perturbation-demodulation loop
• How to catch a lion in the desert: on the solution of the coverage directed generation (CDG) problem
• Extremum Seeking Control with Two-Sided Stochastic Perturbations
• Risk-Sensitive Reinforcement Learning via Policy Gradient Search
• Adaptive optimization algorithm for nonlinear Markov jump systems with partial unknown dynamics
• Truncated Cauchy random perturbations for smoothed functional-based stochastic optimization

This page was built for publication: Adaptive System Optimization Using Random Directions Stochastic Approximation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5280408)