Sample size selection in optimization methods for machine learning
From MaRDI portal
Publication:715253
DOI10.1007/s10107-012-0572-5zbMath1252.49044WikidataQ105583393 ScholiaQ105583393MaRDI QIDQ715253
Gillian M. Chin, Yuchen Wu, Nocedal, Jorge, Byrd, Richard H.
Publication date: 2 November 2012
Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1007/s10107-012-0572-5
65K05: Numerical mathematical programming methods
90C30: Nonlinear programming
68T05: Learning and adaptive systems in artificial intelligence
49M15: Newton-type methods
49M37: Numerical methods based on nonlinear programming
Related Items
Unnamed Item, Adaptive Sampling Strategies for Stochastic Optimization, On Sampling Rates in Simulation-Based Recursions, Stable architectures for deep neural networks, Batched Stochastic Gradient Descent with Weighted Sampling, Variance-Based Extragradient Methods with Line Search for Stochastic Variational Inequalities, Randomized Approach to Nonlinear Inversion Combining Random and Optimized Simultaneous Sources and Detectors, Optimization Methods for Large-Scale Machine Learning, A robust multi-batch L-BFGS method for machine learning, Global Convergence Rate Analysis of a Generic Line Search Algorithm with Noise, Adaptive Deep Learning for High-Dimensional Hamilton--Jacobi--Bellman Equations, A fully stochastic second-order trust region method, A Variable Sample-Size Stochastic Quasi-Newton Method for Smooth and Nonsmooth Stochastic Convex Optimization, Stochastic Trust-Region Methods with Trust-Region Radius Depending on Probabilistic Models, A nonmonotone line search method for stochastic optimization problems, Trust-region algorithms for training responses: machine learning methods using indefinite Hessian approximations, Gradient-Based Adaptive Stochastic Search for Simulation Optimization Over Continuous Space, Asynchronous Schemes for Stochastic and Misspecified Potential Games and Nonconvex Optimization, Convergence of Newton-MR under Inexact Hessian Information, A Stochastic Line Search Method with Expected Complexity Analysis, Solving inverse problems using data-driven models, An Inexact Variable Metric Proximal Point Algorithm for Generic Quasi-Newton Acceleration, A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization, Newton-like Method with Diagonal Correction for Distributed Optimization, Extragradient Method with Variance Reduction for Stochastic Variational Inequalities, LSOS: Line-search second-order stochastic optimization methods for nonconvex finite sums, Probability maximization via Minkowski functionals: convex representations and tractable resolution, An adaptive stochastic sequential quadratic programming with differentiable exact augmented Lagrangians, Inequality constrained stochastic nonlinear optimization via active-set sequential quadratic programming, A trust region method for noisy unconstrained optimization, An adaptive sampling augmented Lagrangian method for stochastic optimization with deterministic constraints, Gradient-based optimisation of the conditional-value-at-risk using the multi-level Monte Carlo method, An overview of stochastic quasi-Newton methods for large-scale machine learning, A framework of convergence analysis of mini-batch stochastic projected gradient methods, Adaptive sampling stochastic multigradient algorithm for stochastic multiobjective optimization, A line search based proximal stochastic gradient algorithm with dynamical variance reduction, Hessian averaging in stochastic Newton methods achieves superlinear convergence, Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization, A Stochastic Quasi-Newton Method for Large-Scale Optimization, An inexact successive quadratic approximation method for L-1 regularized optimization, A family of second-order methods for convex \(\ell _1\)-regularized optimization, Estimating the algorithmic variance of randomized ensembles via the bootstrap, Spectral projected gradient method for stochastic optimization, Convergence of the reweighted \(\ell_1\) minimization algorithm for \(\ell_2-\ell_p\) minimization, Global convergence rate analysis of unconstrained optimization methods based on probabilistic models, Adaptive stochastic approximation algorithm, On variance reduction for stochastic smooth convex optimization with multiplicative noise, Sub-sampled Newton methods, Second-order orthant-based methods with enriched Hessian information for sparse \(\ell _1\)-optimization, Resolving learning rates adaptively by locating stochastic non-negative associated gradient projection points using line searches, On the local convergence of a stochastic semismooth Newton method for nonsmooth nonconvex optimization, Linesearch Newton-CG methods for convex optimization with noise, Accelerating mini-batch SARAH by step size rules, A theoretical and empirical comparison of gradient approximations in derivative-free optimization, Ritz-like values in steplength selections for stochastic gradient methods, Subsampled nonmonotone spectral gradient methods, Inexact restoration with subsampled trust-region methods for finite-sum minimization, A subspace-accelerated split Bregman method for sparse data recovery with joint \(\ell_1\)-type regularizers, A count sketch maximal weighted residual Kaczmarz method for solving highly overdetermined linear systems, A deep learning semiparametric regression for adjusting complex confounding structures, Accelerating deep neural network training with inconsistent stochastic gradient descent, Nonmonotone line search methods with variable sample size, Restricted strong convexity and its applications to convergence analysis of gradient-type methods in convex optimization, Statistically equivalent surrogate material models: impact of random imperfections on the elasto-plastic response, Risk-averse design of tall buildings for uncertain wind conditions, A second-order method for convex1-regularized optimization with active-set prediction, Deep Learning for Trivial Inverse Problems, Parallel Optimization Techniques for Machine Learning, Algorithms for Kullback--Leibler Approximation of Probability Measures in Infinite Dimensions, Descent direction method with line search for unconstrained optimization in noisy environment
Uses Software
Cites Work
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Primal-dual subgradient methods for convex problems
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- An adaptive Monte Carlo algorithm for computing mixed logit estimators
- Variable-number sample-path optimization
- A simulation-based approach to two-stage stochastic programming with recourse
- Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming
- On the Rate of Convergence of Optimal Solutions of Monte Carlo Approximations of Stochastic Programs
- The Sample Average Approximation Method for Stochastic Discrete Optimization
- Accelerated Block-coordinate Relaxation for Regularized Optimization
- On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning
- A Globally Convergent Augmented Lagrangian Algorithm for Optimization with General Constraints and Simple Bounds
- A New Active Set Algorithm for Box Constrained Optimization
- Acceleration of Stochastic Approximation by Averaging
- On the Goldstein-Levitin-Polyak gradient projection method
- Sparse Reconstruction by Separable Approximation
- Newton's Method for Large Bound-Constrained Optimization Problems
- Convergence Analysis of Stochastic Algorithms
- De-noising by soft-thresholding
- Optimal Distributed Online Prediction using Mini-Batches
- The conjugate gradient method in extremal problems
- A Stochastic Approximation Method