Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

DOI10.48550/arXiv.2606.25975MaRDI QIDQ7270461arXivFDO

Authors Vladimir I. Bogachev, M. V. Rakhuba, Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko, Sergei Kudriashov, Maxim Rakhuba

Publication date 24 June 2026

Published in arXiv

Tldr We extend the constrained optimization perspective of Muon to higher-order tensors, introducing Tensorion: a generalization of Muon that balances spectral norm control and tractability through an adaptively selected linear minimization oracle over a carefully chosen tensor norm ball. On computer vision problems, Tensorion offers improved convergence behavior and stable gradient updates compared to baselines.

Full work available at URL https://doi.org/10.48550/arXiv.2606.25975

arXiv classification

cs.CVcs.LGmath.NA

Cites work

• Blind Multilinear Identification
• Concentration Inequalities for Sub-Weibull Random Tensors
• Decoupled Weight Decay Regularization
• Most tensor problems are NP-hard
• Nuclear norm of higher-order tensors
• Operator norm inequalities between tensor unfoldings on the partition lattice
• Optimization on the extended tensor-train manifold with shared factors
• signSGD: Compressed Optimisation for Non-Convex Problems

Cited in

(1)

• Muon on the Stiefel Manifold Admits an Exact Closed-Form Update

This page was built for publication: Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7270461)