AURA: Angular Update Rate Adaptation for training complex-valued neural networks

DOI10.48550/arXiv.2609.25914MaRDI QIDQ7353577arXivFDO

Authors Enrico Ballini, Tito Andriollo, Allan Peter Engsig-Karup

Publication date 22 September 2026

Published in arXiv

Tldr AURA is a per-parameter step-size adaptation for complex-valued neural networks that measures agreement between consecutive updates to enlarge or reduce the update step size based on consistency, alignment, and rotation of complex parameters; AURA improves convergence with small overhead compared to Adam, Muon, and SGD on various test cases.

Full work available at URL https://doi.org/10.48550/arXiv.2609.25914

arXiv classification

cs.LGmath.NA

Cites work

• A holomorphic neural network framework for 3D boundary value problems governed by harmonic potentials
• A meta-cognitive learning algorithm for a fully complex-valued relaxation network
• Accelerated Stochastic Approximation
• AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
• Adam: A Method for Stochastic Optimization
• Adaptive Gradient Method with Resilience and Momentum
• Adaptive subgradient methods for online learning and stochastic optimization
• Calibrating the Adaptive Learning Rate to Improve Convergence of ADAM
• Decoupled Weight Decay Regularization
• diffGrad: An Optimization Method for Convolutional Neural Networks
• Fully complex conjugate gradient-based neural networks using Wirtinger calculus framework: deterministic convergence and its application
• Learning Applied to Successive Approximation Algorithms
• On the Convergence of Adam and Beyond
• Optimization methods for large-scale machine learning
• Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks
• The impact of optimiser choice on the training dynamics of complex-valued neural networks: a comparative study with real-valued counterparts
• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations

This page was built for publication: AURA: Angular Update Rate Adaptation for training complex-valued neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7353577)