On the Exact Computation of Linear Frequency Principle Dynamics and Its Generalization

DOI10.1137/21M1444400MaRDI QIDQ5051354zbMATH OpenOpenAlexFDO

Authors Tao Luo, Zheng Ma, Zhi-Qin John Xu, Yaoyu Zhang

Publication date 23 November 2022

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2010.08153

zbMATH Keywords

deep learningFourier analysisfrequency principleneural tangent kerneloptimizationtwo-layer neural network

Mathematics Subject Classification ID

Function spaces arising in harmonic analysis (42B35) Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• Wide neural networks of any depth evolve as linear models under gradient descent ^*
• On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis
• High-dimensional dynamics of generalization error in neural networks
• Learning high frequency data via the coupled frequency predictor-corrector triangular DNN

Cites work

• A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
• A phase shift deep neural network for high frequency approximation and wave problems
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Approximation by superpositions of a sigmoidal function
• Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
• scientific article; zbMATH DE number 3314813 (Why is no real title available?)
• Machine learning from a continuous viewpoint. I
• Multi-scale deep neural network (MscaleDNN) for solving Poisson-Boltzmann equation in complex domains
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• The vectorization of ITPACK 2C

Cited in

(4)

• On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis
• Loss jump during loss switch in solving PDEs with neural networks
• On understanding and overcoming spectral biases of deep neural network learning methods for solving PDEs
• On spectral bias reduction of multi-scale neural networks for regression problems

This page was built for publication: On the Exact Computation of Linear Frequency Principle Dynamics and Its Generalization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5051354)