Subsampled Hessian Newton Methods for Supervised Learning

DOI10.1162/NECO_A_00751MaRDI QIDQ5380307zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Chien-Chih Wang, Chun-Heng Huang, Chih-Jen Lin

Publication date 4 June 2019

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1162/neco_a_00751

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Large-scale problems in mathematical programming (90C06) Convex programming (90C25)

Recommendations

• Sub-sampled Newton methods
• Subsampled inexact Newton methods for minimizing large sums of convex functions
• Exact and inexact subsampled Newton methods for optimization
• Adaptive iterative Hessian sketch via A-optimal subsampling
• Stochastic sub-sampled Newton method with variance reduction
• Subsampled nonmonotone spectral gradient methods
• Sublinear optimization for machine learning
• Subgradient and sampling algorithms for _1 regression
• Hessian averaging in stochastic Newton methods achieves superlinear convergence
• Optimal subsampling for softmax regression

Cites work

• A finite newton method for classification
• A modified finite Newton method for fast solution of large scale linear SVMs
• A simple automatic derivative evaluation program
• Fast Curvature Matrix-Vector Products for Second-Order Gradient Descent
• scientific article; zbMATH DE number 5359577 (Why is no real title available?)
• scientific article; zbMATH DE number 5430994 (Why is no real title available?)
• scientific article; zbMATH DE number 3579922 (Why is no real title available?)
• scientific article; zbMATH DE number 961607 (Why is no real title available?)
• Iterative scaling and coordinate descent methods for maximum entropy models
• LIBLINEAR: a library for large linear classification
• On the use of stochastic Hessian information in optimization methods for machine learning
• Performance of first-order methods for smooth convex minimization: a novel approach
• Some methods of speeding up the convergence of iteration methods
• Trust Region Methods
• Trust region Newton method for logistic regression

Cited in

(10)

• Sub-sampled Newton methods
• Adaptive iterative Hessian sketch via A-optimal subsampling
• On the use of stochastic Hessian information in optimization methods for machine learning
• Nesterov's acceleration for approximate Newton
• Approximate Newton methods
• Distributed Newton Methods for Deep Neural Networks
• Stochastic sub-sampled Newton method with variance reduction
• Exact and inexact subsampled Newton methods for optimization
• Hessian averaging in stochastic Newton methods achieves superlinear convergence
• On the inversion-free Newton's method and its applications

Describes a project that uses

Uses Software

• LIBLINEAR

This page was built for publication: Subsampled Hessian Newton Methods for Supervised Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380307)