An efficient stochastic Newton algorithm for parameter estimation in logistic regressions

DOI10.1137/19M1261717zbMATH Open1435.62285arXiv1904.07908OpenAlexW2939929204WikidataQ126304926 ScholiaQ126304926MaRDI QIDQ5212017FDOQ5212017

Authors: Bernard Bercu, Antoine Godichon, Bruno Portier 

Publication date: 24 January 2020

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Abstract: Logistic regression is a well-known statistical model which is commonly used in the situation where the output is a binary random variable. It has a wide range of applications including machine learning, public health, social sciences, ecology and econometry. In order to estimate the unknown parameters of logistic regression with data streams arriving sequentially and at high speed, we focus our attention on a recursive stochastic algorithm. More precisely, we investigate the asymptotic behavior of a new stochastic Newton algorithm. It enables to easily update the estimates when the data arrive sequentially and to have research steps in all directions. We establish the almost sure convergence of our stochastic Newton algorithm as well as its asymptotic normality. All our theoretical results are illustrated by numerical experiments.

Full work available at URL: https://arxiv.org/abs/1904.07908

Recommendations

• Quadratic lower-bound algorithm for maximum likelihood estimator of logistic regression on parameter and its application
• A stochastic estimation algorithm with observation averaging
• Stochastic approximation and Newton's estimate of a mixing distribution
• Inversion-free subsampling Newton's method for large sample logistic regression
• Logistic regression with random coefficients

zbMATH Keywords

logistic regressionparametric estimationstochastic Newton algorithm

Mathematics Subject Classification ID

Point estimation (62F10) Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Numerical optimization and variational techniques (65K10) Generalized linear models (logistic models) (62J12) Sequential estimation (62L12)

Cites Work

• A stochastic quasi-Newton method for large-scale optimization
• Acceleration of Stochastic Approximation by Averaging
• Title not available (Why is that?)
• Title not available (Why is that?)
• Asymptotic Almost Sure Efficiency of Averaged Stochastic Algorithms
• A Stochastic Approximation Method
• RES: Regularized Stochastic BFGS Algorithm
• System identification with quantized observations
• Applied logistic regression
• Optimal survey schemes for stochastic gradient descent with applications to \(M\)-estimation
• Online estimation of the asymptotic variance for averaged stochastic gradient algorithms
• Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
• Optimal non-asymptotic analysis of the Ruppert-Polyak averaging stochastic algorithm
• L^p and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective

Cited In (12)

• Deterministic subsampling for logistic regression with massive data
• An Algorithm for Exact Logistic Regression
• Title not available (Why is that?)
• Title not available (Why is that?)
• On the inversion-free Newton's method and its applications
• Inversion-free subsampling Newton's method for large sample logistic regression
• Recursive ridge regression using second-order stochastic algorithms
• An efficient averaged stochastic Gauss-Newton algorithm for estimating parameters of nonlinear regressions models
• A two step algorithm for solving a large scale semi-definite logit model
• Online stochastic Newton methods for estimating the geometric median and applications
• On the asymptotic rate of convergence of stochastic Newton algorithms and their weighted averaged versions
• Multinomial logistic regression algorithm

This page was built for publication: An efficient stochastic Newton algorithm for parameter estimation in logistic regressions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5212017)