A learnable activation function based on orthogonal Legendre wavelets
Fourier series in special orthogonal functions (Legendre polynomials, Walsh functions, etc.) (42C10) Nontrigonometric harmonic analysis involving wavelets and other special systems (42C40) Numerical algorithms for computer arithmetic, etc. (65Y04) Computing methodologies for image processing (68U10)
The paper proposes to use wavelet-like functions based on Legendre orthogonal polynomials as activation functions in neural networks for image classification. It is claimed that, using such an activation function with minor adjustments in the hidden layers, ResNet50 achieved an increase of \(+5.17\%\) in accuracy for CIFAR10 classification. Similarly, in CIFAR100 classification, the authors found that the use of such an activation function in Swin Transformers led to a \(+3.51\%\) boost in accuracy compared to some standard activation functions.
- A new class of radial basis functions with compact support
- Construction and approximation for a class of feedforward neural networks with sigmoidal function
- Frame multiresolution analysis of continuous piecewise linear functions
- Krylov complexity and orthogonal polynomials
- Nonlinear approximation and (deep) ReLU networks
- Numerical solution of nonlinear fractal-fractional optimal control problems by Legendre polynomials
- Proper generalized decomposition using Taylor expansion for non-linear diffusion equations
This page was built for publication: A learnable activation function based on orthogonal Legendre wavelets
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6962152)
