Higher-order reverse automatic differentiation with emphasis on the third-order

DOI10.1007/S10107-014-0827-4zbMATH Open1332.65034arXiv1309.5479OpenAlexW2071726791MaRDI QIDQ5962714FDOQ5962714

Authors: Robert M. Gower, A. L. Gower 

Publication date: 23 February 2016

Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)

Abstract: It is commonly assumed that calculating third order information is too expensive for most applications. But we show that the directional derivative of the Hessian ($D^{3}f(x)cdotd$) can be calculated at a cost proportional to that of a state-of-the-art method for calculating the Hessian matrix. We do this by first presenting a simple procedure for designing high order reverse methods and applying it to deduce several methods including a reverse method that calculates $D^{3}f(x)cdotd$. We have implemented this method taking into account symmetry and sparsity, and successfully calculated this derivative for functions with a million variables. These results indicate that the use of third order information in a general nonlinear solver, such as Halley-Chebyshev methods, could be a practical alternative to Newton's method.

Full work available at URL: https://arxiv.org/abs/1309.5479

Recommendations

• On the Halley class of methods for unconstrained optimization problems
• Computation of high order derivatives in optimal shape design
• Implementation issues for high-order algorithms
• On large-scale unconstrained optimization problems and higher order methods
• Higher-order sensitivity analysis of finite element method by automatic differentiation

zbMATH Keywords

sensitivity analysishigh-order methodsHessian matrixautomatic differentiationtensors vector products

Mathematics Subject Classification ID

Computational methods for sparse matrices (65F50) Multilinear algebra, tensor calculus (15A69) Sensitivity analysis for optimization problems on manifolds (49Q12) Numerical differentiation (65D25)

Cites Work

• The art of differentiating computer programs. An introduction to algorithmic differentiation.
• CUTE
• Evaluating Derivatives
• Test example for nonlinear programming codes
• A family of Chebyshev-Halley type methods in Banach spaces
• Algorithm 755: ADOL-C
• Efficient computation of sparse hessians using coloring and automatic differentiation
• What Color Is Your Jacobian? Graph Coloring for Computing Derivatives
• Third-order iterative methods under Kantorovich conditions
• An efficient method for the numerical evaluation of partial derivatives of arbitrary order
• Third‐order sensitivity analysis for robust aerodynamic design using continuous adjoint
• Semilocal convergence for Halley's method under weak Lipschitz condition
• Semilocal convergence and \(R\)-order for modified Chebyshev-Halley methods
• Reduction of storage requirement by checkpointing for time-dependent optimal control problems in ODEs
• Evaluating higher derivative tensors by forward propagation of univariate Taylor series
• Title not available (Why is that?)
• Newton's method's basins of attraction revisited
• Sparsity in higher order methods for unconstrained optimization
• On large-scale unconstrained optimization problems and higher order methods
• The Newton and Halley Methods for Complex Roots
• On diagonally structured problems in unconstrained optimization using an inexact super Halley method
• New Kantorovich-Type Conditions for Halley's Method
• A new framework for the computation of Hessians
• Title not available (Why is that?)
• Computing Periodic Orbits and their Bifurcations with Automatic Differentiation
• Formulae for high derivatives of composite functions
• Title not available (Why is that?)
• Accumulating Jacobians as chained sparse matrix products
• Higher-order sensitivity analysis of finite element method by automatic differentiation

Cited In (2)

• Title not available (Why is that?)
• Capitalizing on \textit{live} variables: new algorithms for efficient Hessian computation via automatic differentiation

Uses Software

• ADOL-C
• CUTEr
• ColPack
• dcc
• CUTE

This page was built for publication: Higher-order reverse automatic differentiation with emphasis on the third-order

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5962714)