Post-training Quantization for Neural Networks with Provable Guarantees

DOI10.1137/22M1511709MaRDI QIDQ6171681zbMATH OpenOpenAlexFDO

Authors Jinjie Zhang, Rayan Saab

Publication date 14 August 2023

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2201.11113

zbMATH Keywords

deep neural networksgreedy algorithmsquantization of neural networkssparsity

Mathematics Subject Classification ID

Neural nets and related approaches to inference from stochastic processes (62M45) Analysis of algorithms and problem complexity (68Q25) Artificial neural networks and deep learning (68T07) Approximation algorithms (68W25) Convex programming (90C25)

Abstract: While neural networks have been remarkably successful in a wide array of applications, implementing them in resource-constrained hardware remains an area of intense research. By replacing the weights of a neural network with quantized (e.g., 4-bit, or binary) counterparts, massive savings in computation cost, memory, and power consumption are attained. To that end, we generalize a post-training neural-network quantization method, GPFQ, that is based on a greedy path-following mechanism. Among other things, we propose modifications to promote sparsity of the weights, and rigorously analyze the associated error. Additionally, our error analysis expands the results of previous work on GPFQ to handle general quantization alphabets, showing that for quantizing a single-layer network, the relative square error essentially decays linearly in the number of weights -- i.e., level of over-parametrization. Our result holds across a range of input distributions and for both fully-connected and convolutional architectures thereby also extending previous results. To empirically evaluate the method, we quantize several common architectures with few bits per weight, and test them on ImageNet, showing only minor loss of accuracy compared to unquantized models. We also demonstrate that standard modifications, such as bias correction and mixed precision quantization, further improve accuracy.

Recommendations

• A greedy algorithm for quantizing neural networks
• Binary quantized network training with sharpness-aware minimization
• scientific article; zbMATH DE number 6982943
• BinaryRelax: a relaxation approach for training deep neural networks with quantized weights
• Recurrence of optimum for training weight and activation quantized networks

Cites work

• A greedy algorithm for quantizing neural networks

Cited in

(14)

• Quantized convolutional neural networks through the lens of partial differential equations
• Binary quantized network training with sharpness-aware minimization
• Stochastic quantization for learning accurate low-bit deep neural networks
• Evaluations on Deep Neural Networks Training Using Posit Number System
• scientific article; zbMATH DE number 6982943 (Why is no real title available?)
• A greedy algorithm for quantizing neural networks
• Neural Network Precision Tuning
• \textsf{QEBVerif}: quantization error bound verification of neural networks
• Exploring the impact of post-training rounding in regression models.
• Corrigendum to: ``Post-training quantization for neural networks with provable guarantees
• Theoretical guarantees for low-rank compression of deep neural networks
• Frame quantization of neural networks
• Unified stochastic framework for neural network quantization and pruning
• SPFQ: a stochastic algorithm and its error analysis for neural network quantization

This page was built for publication: Post-training Quantization for Neural Networks with Provable Guarantees

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6171681)