An efficient stochastic Newton algorithm for parameter estimation in logistic regressions

DOI10.1137/19M1261717MaRDI QIDQ5212017zbMATH OpenOpenAlexWikidataFDO

Authors Bernard Bercu, Antoine Godichon, Bruno Portier

Publication date 24 January 2020

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1904.07908

zbMATH Keywords

logistic regressionparametric estimationstochastic Newton algorithm

Mathematics Subject Classification ID

Point estimation (62F10) Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Numerical optimization and variational techniques (65K10) Generalized linear models (logistic models) (62J12) Sequential estimation (62L12)

Abstract: Logistic regression is a well-known statistical model which is commonly used in the situation where the output is a binary random variable. It has a wide range of applications including machine learning, public health, social sciences, ecology and econometry. In order to estimate the unknown parameters of logistic regression with data streams arriving sequentially and at high speed, we focus our attention on a recursive stochastic algorithm. More precisely, we investigate the asymptotic behavior of a new stochastic Newton algorithm. It enables to easily update the estimates when the data arrive sequentially and to have research steps in all directions. We establish the almost sure convergence of our stochastic Newton algorithm as well as its asymptotic normality. All our theoretical results are illustrated by numerical experiments.

Recommendations

• Quadratic lower-bound algorithm for maximum likelihood estimator of logistic regression on parameter and its application
• A stochastic estimation algorithm with observation averaging
• Stochastic approximation and Newton's estimate of a mixing distribution
• Inversion-free subsampling Newton's method for large sample logistic regression
• Logistic regression with random coefficients

Cites work

• scientific article; zbMATH DE number 976356 (Why is no real title available?)
• scientific article; zbMATH DE number 1972910 (Why is no real title available?)
• L^p and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective
• A Stochastic Approximation Method
• A stochastic quasi-Newton method for large-scale optimization
• Acceleration of Stochastic Approximation by Averaging
• Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
• Applied logistic regression
• Asymptotic Almost Sure Efficiency of Averaged Stochastic Algorithms
• Online estimation of the asymptotic variance for averaged stochastic gradient algorithms
• Optimal non-asymptotic analysis of the Ruppert-Polyak averaging stochastic algorithm
• Optimal survey schemes for stochastic gradient descent with applications to \(M\)-estimation
• RES: Regularized Stochastic BFGS Algorithm
• System identification with quantized observations

Cited in

(12)

• Deterministic subsampling for logistic regression with massive data
• An Algorithm for Exact Logistic Regression
• scientific article; zbMATH DE number 7626738 (Why is no real title available?)
• scientific article; zbMATH DE number 5260684 (Why is no real title available?)
• On the inversion-free Newton's method and its applications
• Inversion-free subsampling Newton's method for large sample logistic regression
• Recursive ridge regression using second-order stochastic algorithms
• An efficient averaged stochastic Gauss-Newton algorithm for estimating parameters of nonlinear regressions models
• A two step algorithm for solving a large scale semi-definite logit model
• Multinomial logistic regression algorithm
• On the asymptotic rate of convergence of stochastic Newton algorithms and their weighted averaged versions
• Online stochastic Newton methods for estimating the geometric median and applications

This page was built for publication: An efficient stochastic Newton algorithm for parameter estimation in logistic regressions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5212017)