Activations and gradients compression for model-parallel training

DOI10.1134/S1064562423701314MaRDI QIDQ6204274zbMATH OpenFDO

Authors M. I. Rudakov, A. N. Beznosikov, Ya. A. Kholodov, A. V. Gasnikov

Publication date 27 March 2024

Published in Doklady Mathematics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2401.07788

zbMATH Keywords

activation compressiondistributed learningerror feedbackgradient compressionmodel parallelism

Mathematics Subject Classification ID

Computer science (68-XX) Game theory, economics, finance, and other social and behavioral sciences (91-XX)

This page was built for publication: Activations and gradients compression for model-parallel training

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6204274)