Polar Switch: Fast Muon Orthogonalization with Polynomial Circuits

DOI10.48550/ARXIV.2610.26742MaRDI QIDQ7400226arXivFDO

Authors Zhehang Du, Weijie Su

Publication date 19 October 2026

Published in arXiv

Tldr We introduce Polar Switch, a simple circuit-based approach for fast polynomial gradient orthogonalization with lower FLOP cost than quintic iterations. We design and numerically minimize the worst-case polar approximation error under an FLOP budget to reach desired polynomials.

Full work available at URL https://doi.org/10.48550/ARXIV.2610.26742

arXiv classification

cs.LGmath.NAmath.OC

Cites work

• A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
• A new analysis of iterative refinement and its application to accurate solution of ill-conditioned sparse linear systems
• A Note on the Convergence of Muon
• A Schur-Padé algorithm for fractional powers of a matrix
• A Schur-Parlett Algorithm for Computing Matrix Functions
• Accelerating Newton-Schulz Iteration for Orthogonalization via Chebyshev-type Polynomials
• Accelerating scientific computations with mixed precision algorithms
• Adaptive subgradient methods for online learning and stochastic optimization
• An Iterative Algorithm for Computing the Best Estimate of an Orthogonal Matrix
• Backward stability of iterations for computing the polar decomposition
• Barycentric-Remez algorithms for best polynomial approximation in the chebfun system
• Bernstein's inequality for multivariate polynomials on the standard simplex
• Computing Fundamental Matrix Decompositions Accurately via the Matrix Sign Function in Two Iterations: The Power of Zolotarev's Functions
• Computing real square roots of a real matrix
• Computing the action of the matrix exponential, with an application to exponential integrators
• Computing the Polar Decomposition—with Applications
• Decoupled Weight Decay Regularization
• Efficient evaluation of matrix polynomials
• Efficient Parallel Evaluation of Straight-Line Code and Arithmetic Circuits
• Functions of Matrices
• Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization
• High-Order Retractions on Matrix Manifolds Using Projected Polynomials
• Linear model reduction and solution of the algebraic Riccati equation by use of the sign function†
• On the Newton Method for the Matrix Pth Root
• On the Number of Nonscalar Multiplications Necessary to Evaluate Polynomials
• Optimizing Halley's Iteration for Computing the Matrix Polar Decomposition
• Rational Iterative Methods for the Matrix Sign Function
• Rational Krylov approximation of matrix functions: numerical methods and optimal pole selection
• Rational minimax approximation via adaptive barycentric representations
• Shampoo: Preconditioned Stochastic Tensor Optimization
• Small singular values can increase in lower precision
• Stable and efficient spectral divide and conquer algorithms for the symmetric eigenvalue decomposition and the SVD
• Stable iterations for the matrix square root
• The AAA algorithm for rational approximation
• The matrix sign function and computations in systems
• The Polynomial Set Associated with a Fixed Number of Matrix-Matrix Multiplications
• The scaling and squaring method for the matrix exponential revisited
• Uniform approximation of \(\text{sgn} (x)\) by polynomials and entire functions

This page was built for publication: Polar Switch: Fast Muon Orthogonalization with Polynomial Circuits

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7400226)