On the limited memory BFGS method for large scale optimization

DOI10.1007/BF01589116zbMath0696.90048WikidataQ57311064 ScholiaQ57311064MaRDI QIDQ911463

Dong C. Liu, Nocedal, Jorge

Publication date: 1989

Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)

zbMATH Keywords

convergence analysisconjugate gradient methodscomputational studycomparison of algorithmslarge scale optimizationnumerical performancepartitioned quasi-Newton methodscaling effectssmooth unconstrained problems

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Large-scale problems in mathematical programming (90C06) Nonlinear programming (90C30) Methods of reduced gradient type (90C52)

Related Items (only showing first 100 items - show all)

Iterated dynamic neighborhood search for packing equal circles on a sphere ⋮ Numerical analysis of the model of optimal savings and borrowing ⋮ A restart scheme for the memoryless BFGS method ⋮ Solution of the inverse problem of calculating a gas-liquid injector with a two-phase flow ⋮ Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions ⋮ Automatic boundary fitting framework of boundary dependent physics-informed neural network solving partial differential equation with complex boundary conditions ⋮ A symmetry group based supervised learning method for solving partial differential equations ⋮ Probabilistic Registration for Gaussian Process Three-Dimensional Shape Modelling in the Presence of Extensive Missing Data ⋮ A priori error analysis of linear and nonlinear periodic Schrödinger equations with analytic potentials ⋮ Bayesian combinatorial multistudy factor analysis ⋮ Revisiting Convolutional Neural Networks from the Viewpoint of Kernel-Based Methods ⋮ Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations ⋮ Continuous Newton-like Methods Featuring Inertia and Variable Mass ⋮ On the global minimum of the classical potential energy for clusters bound by many-body forces ⋮ Hyperspectral super-resolution via low rank tensor triple decomposition ⋮ Gradient-enhanced physics-informed neural networks based on transfer learning for inverse problems of the variable coefficient differential equations ⋮ A log-Gaussian Cox process with sequential Monte Carlo for line narrowing in spectroscopy ⋮ Optimal Dirichlet boundary control by Fourier neural operators applied to nonlinear optics ⋮ Robust autoencoder feature selector for unsupervised feature selection ⋮ Many-stage optimal stabilized Runge-Kutta methods for hyperbolic partial differential equations ⋮ Learning particle swarming models from data with Gaussian processes ⋮ Multilevel domain decomposition-based architectures for physics-informed neural networks ⋮ Be greedy and learn: efficient and certified algorithms for parametrized optimal control problems ⋮ Numerical approximation of Dynkin games with asymmetric information ⋮ PDE-READ: human-readable partial differential equation discovery using deep learning ⋮ Certifying optimality of Bell inequality violations: noncommutative polynomial optimization through semidefinite programming and local optimization ⋮ Active learning for regression by inverse distance weighting ⋮ I-FENN with temporal convolutional networks: expediting the load-history analysis of non-local gradient damage propagation ⋮ Two-stage initial-value iterative physics-informed neural networks for simulating solitary waves of nonlinear wave equations ⋮ On quasi-Newton methods in fast Fourier transform-based micromechanics ⋮ Data-driven fusion and fission solutions in the Hirota-Satsuma-Ito equation via the physics-informed neural networks method ⋮ \texttt{Weak-PDE-LEARN}: a weak form based approach to discovering PDEs from noisy, limited data ⋮ A structured L-BFGS method and its application to inverse problems ⋮ Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning ⋮ An adaptive discrete physics-informed neural network method for solving the Cahn-Hilliard equation ⋮ Convergence analysis of block majorize-minimize subspace approach ⋮ Sparse signal reconstruction via Hager–Zhang-type schemes for constrained system of nonlinear equations ⋮ An efficient solution space exploring and descent method for packing equal spheres in a sphere ⋮ Identification of reaction rate parameters from uncertain spatially distributed concentration data using gradient-based PDE constrained optimization ⋮ Meshless physics-informed deep learning method for three-dimensional solid mechanics ⋮ \(PT\)-symmetric PINN for integrable nonlocal equations: forward and inverse problems ⋮ Adaptive sampling physics-informed neural network method for high-order rogue waves and parameters discovery of the \((2+1)\)-dimensional CHKP equation ⋮ A probabilistic framework for multidisciplinary design: application to the hydrostructural optimization of supercavitating hydrofoils ⋮ On the convergence of inexact alternate minimization in problems with \(\ell_0\) penalties ⋮ An optimized CIP-FEM to reduce the pollution errors for the Helmholtz equation on a general unstructured mesh ⋮ Data driven soliton solution of the nonlinear Schrödinger equation with certain \(\mathcal{PT}\)-symmetric potentials via deep learning ⋮ The appeals of quadratic majorization-minimization ⋮ A performance analysis of Basin Hopping compared to established metaheuristics for global optimization ⋮ The line rogue wave solutions of the nonlocal Davey-Stewartson I equation with \textit{PT} symmetry based on the improved physics-informed neural network ⋮ Uniform flow in axisymmetric devices through permeability optimization ⋮ Subgradient regularized multivariate convex regression at scale ⋮ PINN enhanced extended multiscale finite element method for fast mechanical analysis of heterogeneous materials ⋮ Another Hager-Zhang-type method via singular-value study for constrained monotone equations with application ⋮ A gradient-enhanced physics-informed neural networks method for the wave equation ⋮ A branch-and-bound algorithm with growing datasets for large-scale parameter estimation ⋮ Training adaptive reconstruction networks for blind inverse problems ⋮ Phase-field modeling of fracture with physics-informed deep learning ⋮ Variational temporal convolutional networks for I-FENN thermoelasticity ⋮ Phase field smoothing-PINN: a neural network solver for partial differential equations with discontinuous coefficients ⋮ Extremization to fine tune physics informed neural networks for solving boundary value problems ⋮ Discontinuous polynomial approximation in electrical impedance tomography with total variational regularization ⋮ Hypergraph-based convex semi-supervised unconstraint symmetric matrix factorization for image clustering ⋮ Exact enforcement of temporal continuity in sequential physics-informed neural networks ⋮ Physics-specialized neural network with hard constraints for solving multi-material diffusion problems ⋮ Skyrmion crystals in the triangular Kondo lattice model ⋮ Particle swarm optimization for searching efficient experimental designs: a review ⋮ New subspace method for unconstrained derivative-free optimization ⋮ Interpretable physics-encoded finite element network to handle concentration features and multi-material heterogeneity in hyperelasticity ⋮ Generalized functional linear model with a point process predictor ⋮ A short note on solving partial differential equations using convolutional neural networks ⋮ Enhancing training of physics-informed neural networks using domain decomposition-based preconditioning strategies ⋮ Short-time large deviations of the spatially averaged height of a Kardar-Parisi-Zhang interface on a ring ⋮ Quantitative analysis of nonlinear multifidelity optimization for inverse electrophysiology ⋮ Robust Low-Rank Tensor Decomposition with the L ₂ Criterion ⋮ Function-on-Function Kriging, With Applications to Three-Dimensional Printing of Aortic Tissues ⋮ Convergent least-squares optimization methods for variational data assimilation ⋮ Novel localized wave of modified Kadomtsev-Petviashvili equation ⋮ An extended version of the memoryless DFP algorithm with the sufficient descent property ⋮ No dimension-free deterministic algorithm computes approximate stationarities of Lipschitzians ⋮ Tuning parameters of deep neural network training algorithms pays off: a computational study ⋮ Ensemble of physics-informed neural networks for solving plane elasticity problems with examples ⋮ A probability constrained dynamic switching optimization method for the energy dispatch strategy of hybrid power systems with renewable energy resources and uncertainty ⋮ Effective matrix adaptation strategy for noisy derivative-free optimization ⋮ A novel phase-field monolithic scheme for brittle crack propagation based on the limited-memory BFGS method with adaptive mesh refinement ⋮ Quantum phases of hardcore bosons with repulsive dipolar density-density interactions on two-dimensional lattices ⋮ Prediction of spatiotemporal dynamics using deep learning: coupled neural networks of long short-terms memory, auto-encoder and physics-informed neural networks ⋮ A proximal stochastic quasi-Newton algorithm with dynamical sampling and stochastic line search ⋮ A family of limited memory three term conjugate gradient methods ⋮ I-FENN for thermoelasticity based on physics-informed temporal convolutional network (PI-TCN) ⋮ Structured optimization-based model order reduction for parametric systems ⋮ SketchySGD: reliable stochastic optimization via randomized curvature estimates ⋮ A limited memory quasi-Newton trust-region method for box constrained optimization ⋮ Solving forward and inverse problems of the logarithmic nonlinear Schrödinger equation with \(\mathcal{PT}\)-symmetric harmonic potential via deep learning ⋮ Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization ⋮ Herd immunity levels and multi-strain influenza epidemics in Russia: a modelling study ⋮ Hessian initialization strategies for \(\ell \)-BFGS solving non-linear inverse problems ⋮ Variational models for signal processing with graph neural networks ⋮ A hybrid random field model for scalable statistical learning ⋮ A limited memory BFGS algorithm for non-convex minimization with applications in matrix largest eigenvalue problem ⋮ Lifted generative learning of Markov logic networks

Uses Software

• Unnamed Item
• L-BFGS
• tn
• ve08
• Algorithm 500
• minpack

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Some numerical experiments with variable-storage quasi-Newton algorithms
• Partitioned variable metric updates for large structured optimization problems
• Local convergence analysis for partitioned quasi-Newton updates
• The Conjugate Gradient Method and Trust Regions in Large Scale Optimization
• QN-like variable storage conjugate gradients
• Algorithm 630
• Preconditioning of Truncated-Newton Methods
• A Tool for the Analysis of Quasi-Newton Methods with Application to Unconstrained Minimization
• A Relationship between the BFGS and Conjugate Gradient Algorithms and Its Implications for New Algorithms
• Testing Unconstrained Optimization Software
• Updating Quasi-Newton Matrices with Limited Storage
• A discrete Newton algorithm for minimizing a function of many variables
• Matrix conditioning and nonlinear optimization
• Some Numerical Results Using a Sparse Matrix Updating Formula in Unconstrained Optimization
• A combined conjugate-gradient quasi-Newton minimization algorithm
• Restart procedures for the conjugate gradient method
• Conjugate Gradient Methods with Inexact Searches
• On the Convergence of a New Conjugate Gradient Algorithm
• Variable metric methods of minimisation

This page was built for publication: On the limited memory BFGS method for large scale optimization