Reproducing activation function for deep learning

DOI10.4310/CMS.2024.V22.N2.A1MaRDI QIDQ6558612zbMATH OpenWikidataFDO

Authors Senwei Liang, Liyao Lyu, ChunMei Wang, Haizhao Yang

Publication date 19 June 2024

Published in Communications in Mathematical Sciences (Search for Journal in Brave)

Full work available at URL https://doi.org/10.4310/CMS.2024.V22.N2.A1

zbMATH Keywords

activation functiondeep neural networkFourier basisneural tangent kernelpolynomialsradial basis functionswavelets

Mathematics Subject Classification ID

Numerical approximation and computational geometry (primarily algorithms) (65D99) Artificial neural networks and deep learning (68T07)

Recommendations

• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Constructive deep ReLU neural network approximation
• Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks
• A representer theorem for deep neural networks
• Generalised rational approximation and its application to improve deep learning classifiers

Cited in

(4)

• Forecasting of spatiotemporal nonlinear dynamic systems by physics-informed neural networks with ResNet blocks
• Learnable activation functions in physics-informed neural networks for solving partial differential equations
• Overview frequency principle/spectral bias in deep learning
• Multiscale finite expression method for PDEs with oscillatory solutions on complex domains

This page was built for publication: Reproducing activation function for deep learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6558612)