Stochastic Markov Gradient Descent and Training Low-Bit Neural Networks

DOI10.1007/S43670-021-00015-1MaRDI QIDQ6347718arXivFDO

Authors Jonathan Ashbrock, Alexander M. Powell

Publication date 25 August 2020

Published in arXiv

Full work available at URL https://doi.org/10.1007/S43670-021-00015-1

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Learning and adaptive systems in artificial intelligence (68T05) Nonconvex programming, global optimization (90C26)

arXiv classification

cs.LGmath.OCstat.ML

Abstract: The massive size of modern neural networks has motivated substantial recent interest in neural network quantization. We introduce Stochastic Markov Gradient Descent (SMGD), a discrete optimization method applicable to training quantized neural networks. The SMGD algorithm is designed for settings where memory is highly constrained during training. We provide theoretical guarantees of algorithm performance as well as encouraging numerical results.

This page was built for publication: Stochastic Markov Gradient Descent and Training Low-Bit Neural Networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6347718)