Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence

DOI10.48550/arXiv.2608.22129MaRDI QIDQ7285021arXivFDO

Authors Rodion Podorozhny

Publication date 22 August 2026

Published in arXiv

Tldr We introduce a blockwise adaptive cubic regularizer with solvers via recurrence that remains exact on every block, and prove the $\mathcal{O}(ε^{-3/2})$ iteration complexity bound and second-order guarantee; this scales to large models and achieves state-of-the-art performance in implicit neural representations.

Full work available at URL https://doi.org/10.48550/arXiv.2608.22129

arXiv classification

cs.LGmath.NA

Cites work

• A General Recipe for Parameter-Free Nonconvex Optimization via Higher-Order Regularization
• A MINRES-based Linesearch Algorithm for Nonconvex Optimization with Non-positive Curvature Detection
• A Multilevel Low-Rank Newton Method with Super-linear Convergence Rate and its Application to Non-convex Problems
• A Novel Fast Exact Subproblem Solver for Stochastic Quasi-Newton Cubic Regularized Optimization
• A scalable sequential adaptive cubic regularization algorithm for optimization with general equality constraints
• A simple and practical adaptive trust-region method
• Acceleration Methods
• Adaptive Newton-CG methods with global and local analysis for unconstrained optimization with Hölder continuous Hessian
• Adaptive Regularization within Trust Region Methods for Stochastic Nonconvex Optimization
• Adaptive Regularized Newton Method with Inexact Hessian
• Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems
• Analysis of Some Krylov Subspace Approximations to the Matrix Exponential Operator
• Block cubic Newton with greedy selection
• Chebyshev-Exact Acceleration under Hessian Variation, I: Sine-Jacobi Method
• Condition numbers and equilibration of matrices
• Cubic regularization of Newton method and its global performance
• Decoupled Weight Decay Regularization
• Exponential integrators
• Extended-Krylov-subspace methods for trust-region and norm-regularization subproblems
• Fast Convergence to Second-Order Stationary Point through Random Subspace Optimization
• Federated cubic regularized Newton learning with sparsification-amplified differential privacy
• First and zeroth-order implementations of the regularized Newton method with lazy approximated Hessians
• First- and Second-Order Stochastic Adaptive Regularization with Cubics: High Probability Iteration and Sample Complexity
• Gradient descent finds the cubic-regularized nonconvex Newton step
• How Does Adaptive Optimization Impact Local Neural Network Geometry?
• Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
• Inexact tensor methods and their application to stochastic convex optimization
• Iterative Methods for Sparse Linear Systems
• Lower bounds for finding stationary points I
• Matrix Computations
• Newton-type methods for non-convex optimization under inexact Hessian information
• On Global Rates for Regularization Methods based on Secant Derivative Approximations
• On Krylov Subspace Approximations to the Matrix Exponential Operator
• On Nesterov's smooth Chebyshev-Rosenbrock function
• On the complexity of steepest descent, Newton's and regularized Newton's methods for nonconvex unconstrained optimization problems
• Polynomial Preconditioning for Gradient Methods
• Randomized Block Cubic Newton Method
• Randomized subspace regularized Newton method for unconstrained non-convex optimization
• Re^3MCN: Cubic Newton + Variance Reduction + Momentum + Quadratic Regularization for Finite-sum Non-convex Problems
• Regularized methods via cubic model subspace minimization for nonconvex optimization
• Riemannian adaptive regularized Newton methods with Hölder continuous Hessians
• Scalable adaptive cubic regularization methods
• Scalable Second-Order Optimization Algorithms for Minimizing Low-rank Functions
• Second-order optimization with lazy Hessians
• Solving the Trust-Region Subproblem using the Lanczos Method
• Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions
• Stochastic Cubic Regularization for Fast Nonconvex Optimization
• Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function
• Sub-sampled Cubic Regularization for Non-convex Optimization
• Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods
• Universal Reduced-Operator Method and High-Order Global Curvature Bounds

This page was built for publication: Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7285021)