Conjugate priors and bias reduction for logistic regression models

DOI10.1016/J.SPL.2023.109901arXiv2202.08734MaRDI QIDQ6178693FDOQ6178693

Authors: Tommaso Rigon, Emanuele Aliverti 

Publication date: 4 September 2023

Published in: Statistics \& Probability Letters (Search for Journal in Brave)

Abstract: Logistic regression models for binomial responses are routinely used in statistical practice. However, the maximum likelihood estimate may not exist due to data separability. We address this issue by considering a conjugate prior penalty which always produces finite estimates. Such a specification has a clear Bayesian interpretation and enjoys several invariance properties, making it an appealing prior choice. We show that the proposed method leads to an accurate approximation of the reduced-bias approach of Firth (1993), resulting in estimators with smaller asymptotic bias than the maximum-likelihood and whose existence is always guaranteed. Moreover, the considered penalized likelihood can be expressed as a genuine likelihood, in which the original data are replaced with a collection of pseudo-counts. Hence, our approach may leverage well established and scalable algorithms for logistic regression. We compare our estimator with alternative reduced-bias methods, vastly improving their computational performance and achieving appealing inferential results.

Full work available at URL: https://arxiv.org/abs/2202.08734

zbMATH Keywords

conjugate priorbias reductionexponential familyboundary estimatepseudo-counts

Mathematics Subject Classification ID

Point estimation (62F10) Bayesian inference (62F15) Generalized linear models (logistic models) (62J12) Characterization and structure theory of statistical distributions (62E10)

Cites Work

• Mean and median bias reduction in generalized linear models
• Title not available (Why is that?)
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• A generic algorithm for reducing bias in parametric estimation
• On the existence of maximum likelihood estimates in logistic regression models
• Title not available (Why is that?)
• Bias reduction of maximum likelihood estimates
• Median bias reduction of maximum likelihood estimates
• Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models
• Conjugate priors for exponential families
• Intrinsic losses
• THE ESTIMATION AND SIGNIFICANCE OF THE LOGARITHM OF A RATIO OF FREQUENCIES
• On estimating binomial response relations
• Conditionally conjugate mean-field variational Bayes for logistic models
• Title not available (Why is that?)
• Generalized conjugate priors for Bayesian analysis of risk and survival regressions
• A modern maximum-likelihood theory for high-dimensional logistic regression

Cited In (4)

• A weakly informative default prior distribution for logistic and other regression models
• Title not available (Why is that?)
• Logistic regression with weight grouping priors
• Conditionally conjugate mean-field variational Bayes for logistic models

This page was built for publication: Conjugate priors and bias reduction for logistic regression models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6178693)