Conjugate priors and bias reduction for logistic regression models

DOI10.1016/J.SPL.2023.109901MaRDI QIDQ6178693FDO

Authors Tommaso Rigon, Emanuele Aliverti

Publication date 4 September 2023

Published in Statistics \& Probability Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2202.08734

zbMATH Keywords

conjugate priorbias reductionexponential familyboundary estimatepseudo-counts

Mathematics Subject Classification ID

Point estimation (62F10) Bayesian inference (62F15) Generalized linear models (logistic models) (62J12) Characterization and structure theory of statistical distributions (62E10)

Abstract: Logistic regression models for binomial responses are routinely used in statistical practice. However, the maximum likelihood estimate may not exist due to data separability. We address this issue by considering a conjugate prior penalty which always produces finite estimates. Such a specification has a clear Bayesian interpretation and enjoys several invariance properties, making it an appealing prior choice. We show that the proposed method leads to an accurate approximation of the reduced-bias approach of Firth (1993), resulting in estimators with smaller asymptotic bias than the maximum-likelihood and whose existence is always guaranteed. Moreover, the considered penalized likelihood can be expressed as a genuine likelihood, in which the original data are replaced with a collection of pseudo-counts. Hence, our approach may leverage well established and scalable algorithms for logistic regression. We compare our estimator with alternative reduced-bias methods, vastly improving their computational performance and achieving appealing inferential results.

Cites work

• scientific article; zbMATH DE number 47310 (Why is no real title available?)
• scientific article; zbMATH DE number 469130 (Why is no real title available?)
• scientific article; zbMATH DE number 1932867 (Why is no real title available?)
• A generic algorithm for reducing bias in parametric estimation
• A modern maximum-likelihood theory for high-dimensional logistic regression
• Bias reduction of maximum likelihood estimates
• Conditionally conjugate mean-field variational Bayes for logistic models
• Conjugate priors for exponential families
• Generalized conjugate priors for Bayesian analysis of risk and survival regressions
• Intrinsic losses
• Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models
• Mean and median bias reduction in generalized linear models
• Median bias reduction of maximum likelihood estimates
• On estimating binomial response relations
• On the existence of maximum likelihood estimates in logistic regression models
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• THE ESTIMATION AND SIGNIFICANCE OF THE LOGARITHM OF A RATIO OF FREQUENCIES

Cited in

(4)

• Logistic regression with weight grouping priors
• scientific article; zbMATH DE number 1932867 (Why is no real title available?)
• A weakly informative default prior distribution for logistic and other regression models
• Conditionally conjugate mean-field variational Bayes for logistic models

This page was built for publication: Conjugate priors and bias reduction for logistic regression models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6178693)