The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression

DOI10.1214/18-AOS1789MaRDI QIDQ2176606zbMATH OpenOpenAlexFDO

Authors Emmanuel J. Candès, Pragya Sur

Publication date 5 May 2020

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1804.09753, https://projecteuclid.org/euclid.aos/1581930124

zbMATH Keywords

decision surfacehigh-dimensional logistic regressionmaximum likelihood estimate (MLE) phase transitionmultivariate centered normal distribution

Mathematics Subject Classification ID

Multivariate distribution of statistics (62H10) Asymptotic distribution theory in statistics (62E20) Generalized linear models (logistic models) (62J12) General considerations in statistical decision theory (62C05)

Abstract: This paper rigorously establishes that the existence of the maximum likelihood estimate (MLE) in high-dimensional logistic regression models with Gaussian covariates undergoes a sharp `phase transition'. We introduce an explicit boundary curve $h_{extMLE}$, parameterized by two scalars measuring the overall magnitude of the unknown sequence of regression coefficients, with the following property: in the limit of large sample sizes $n$ and number of features $p$ proportioned in such a way that $p/nightarrowkappa$, we show that if the problem is sufficiently high dimensional in the sense that $kappa>h_{extMLE}$, then the MLE does not exist with probability one. Conversely, if $kappa<h_{extMLE}$, the MLE asymptotically exists with probability one.

Recommendations

• The existence of maximum likelihood estimate in high-dimensional binary response generalized linear models
• A modern maximum-likelihood theory for high-dimensional logistic regression
• Rate of convergence of the probability of non-existence of the MLE's in simple logistic regression
• Sparse high-dimensional linear regression. Estimating squared error and a phase transition
• Maximum likelihood estimation in logistic regression models with a diverging number of covariates

Cites work

• scientific article; zbMATH DE number 3976123 (Why is no real title available?)
• scientific article; zbMATH DE number 4061904 (Why is no real title available?)
• scientific article; zbMATH DE number 4096600 (Why is no real title available?)
• scientific article; zbMATH DE number 3746271 (Why is no real title available?)
• scientific article; zbMATH DE number 47310 (Why is no real title available?)
• A modern maximum-likelihood theory for high-dimensional logistic regression
• A new perspective on least squares under convex constraint
• A note on A. Albert and J. A. Anderson's conditions for the existence of maximum likelihood estimates in logistic regression models
• Enumeration of Seven-Argument Threshold Functions
• Infinite Parameter Estimates in Logistic Regression, with Application to Approximate Conditional Inference
• Living on the edge: phase transitions in convex programs with random data
• Measuring overlap in binary regression.
• On existence and uniqueness of a vector minimizing a convex function
• On the existence of maximum likelihood estimates in logistic regression models
• Stochastic and Integral Geometry
• The likelihood ratio test in high-dimensional logistic regression is asymptotically a rescaled Chi-square
• Universality laws for randomized dimension reduction, with applications

Cited in

(28)

• Large dimensional analysis of general margin based classification methods
• Stochastic Tverberg theorems with applications in multiclass logistic regression, separability, and centerpoints of data
• Computationally efficient likelihood inference in exponential families when the maximum likelihood estimator does not exist
• A Scale-Free Approach for False Discovery Rate Control in Generalized Linear Models
• scientific article; zbMATH DE number 7306882 (Why is no real title available?)
• Statistics of robust optimization: a generalized empirical likelihood approach
• A modern maximum-likelihood theory for high-dimensional logistic regression
• On functional logistic regression: some conceptual issues
• A Unifying Tutorial on Approximate Message Passing
• Learning curves of generic features maps for realistic datasets with a teacher-student model*
• Rate of convergence of the probability of non-existence of the MLE's in simple logistic regression
• Bounded-memory adjusted scores estimation in generalized linear models with large data sets
• Robust estimation of a regression function in exponential families
• Phase transitions for the uniform distribution in the pattern maximum likelihood problem and its Bethe approximation
• The asymptotic distribution of the MLE in high-dimensional logistic models: arbitrary covariance
• Maximum likelihood estimation in logistic regression models with a diverging number of covariates
• scientific article; zbMATH DE number 7625184 (Why is no real title available?)
• Automatic bias correction for testing in high‐dimensional linear models
• Sparse high-dimensional linear regression. Estimating squared error and a phase transition
• Precise statistical analysis of classification accuracies for adversarial training
• The existence of maximum likelihood estimate in high-dimensional binary response generalized linear models
• scientific article; zbMATH DE number 7370646 (Why is no real title available?)
• Tractability from overparametrization: the example of the negative perceptron
• Generalisation error in learning with random features and the hidden manifold model*
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• Large scale analysis of generalization error in learning using margin based classification methods
• A discussion of ``A note on universal inference by Tse and Davison
• A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers

This page was built for publication: The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2176606)