Stabilize deep ResNet with a sharp scaling factor

DOI10.1007/S10994-022-06192-XMaRDI QIDQ2102389zbMATH OpenOpenAlexFDO

Authors Huishuai Zhang, Da Yu, Mingyang Yi, Wei Chen, Tie-Yan Liu

Publication date 28 November 2022

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/s10994-022-06192-x

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Cites work

• A mean field view of the landscape of two-layer neural networks
• Adaptive estimation of a quadratic functional by model selection.
• Convex Formulation of Overparameterized Deep Neural Networks
• Gradient descent optimizes over-parameterized deep ReLU networks
• Smoothed analysis of algorithms
• Stable architectures for deep neural networks

Cited in

(1)

• Scaling ResNets in the large-depth regime

Describes a project that uses

Uses Software

• Tensor2Tensor
• BERT
• CIFAR

This page was built for publication: Stabilize deep ResNet with a sharp scaling factor \(\tau\)

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2102389)