The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression

DOI10.1214/18-AOS1789zbMATH Open1439.62171arXiv1804.09753OpenAlexW3007237875MaRDI QIDQ2176606FDOQ2176606

Authors: Emmanuel J. Candès, Pragya Sur 

Publication date: 5 May 2020

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This paper rigorously establishes that the existence of the maximum likelihood estimate (MLE) in high-dimensional logistic regression models with Gaussian covariates undergoes a sharp `phase transition'. We introduce an explicit boundary curve $h_{extMLE}$, parameterized by two scalars measuring the overall magnitude of the unknown sequence of regression coefficients, with the following property: in the limit of large sample sizes $n$ and number of features $p$ proportioned in such a way that $p/nightarrowkappa$, we show that if the problem is sufficiently high dimensional in the sense that $kappa>h_{extMLE}$, then the MLE does not exist with probability one. Conversely, if $kappa<h_{extMLE}$, the MLE asymptotically exists with probability one.

Full work available at URL: https://arxiv.org/abs/1804.09753

Recommendations

• The existence of maximum likelihood estimate in high-dimensional binary response generalized linear models
• A modern maximum-likelihood theory for high-dimensional logistic regression
• Rate of convergence of the probability of non-existence of the MLE's in simple logistic regression
• Sparse high-dimensional linear regression. Estimating squared error and a phase transition
• Maximum likelihood estimation in logistic regression models with a diverging number of covariates

zbMATH Keywords

decision surfacehigh-dimensional logistic regressionmaximum likelihood estimate (MLE) phase transitionmultivariate centered normal distribution

Mathematics Subject Classification ID

Multivariate distribution of statistics (62H10) Asymptotic distribution theory in statistics (62E20) Generalized linear models (logistic models) (62J12) General considerations in statistical decision theory (62C05)

Cites Work

• Title not available (Why is that?)
• On the existence of maximum likelihood estimates in logistic regression models
• Measuring overlap in binary regression.
• Stochastic and Integral Geometry
• Enumeration of Seven-Argument Threshold Functions
• Title not available (Why is that?)
• Living on the edge: phase transitions in convex programs with random data
• A note on A. Albert and J. A. Anderson's conditions for the existence of maximum likelihood estimates in logistic regression models
• Title not available (Why is that?)
• A new perspective on least squares under convex constraint
• Infinite Parameter Estimates in Logistic Regression, with Application to Approximate Conditional Inference
• On existence and uniqueness of a vector minimizing a convex function
• Title not available (Why is that?)
• Title not available (Why is that?)
• Universality laws for randomized dimension reduction, with applications
• The likelihood ratio test in high-dimensional logistic regression is asymptotically a rescaled Chi-square
• A modern maximum-likelihood theory for high-dimensional logistic regression

Cited In (28)

• Stochastic Tverberg theorems with applications in multiclass logistic regression, separability, and centerpoints of data
• A Scale-Free Approach for False Discovery Rate Control in Generalized Linear Models
• Title not available (Why is that?)
• Computationally efficient likelihood inference in exponential families when the maximum likelihood estimator does not exist
• Statistics of robust optimization: a generalized empirical likelihood approach
• A modern maximum-likelihood theory for high-dimensional logistic regression
• On functional logistic regression: some conceptual issues
• A Unifying Tutorial on Approximate Message Passing
• Learning curves of generic features maps for realistic datasets with a teacher-student model*
• Rate of convergence of the probability of non-existence of the MLE's in simple logistic regression
• Bounded-memory adjusted scores estimation in generalized linear models with large data sets
• Robust estimation of a regression function in exponential families
• Phase transitions for the uniform distribution in the pattern maximum likelihood problem and its Bethe approximation
• The asymptotic distribution of the MLE in high-dimensional logistic models: arbitrary covariance
• Title not available (Why is that?)
• Maximum likelihood estimation in logistic regression models with a diverging number of covariates
• Automatic bias correction for testing in high‐dimensional linear models
• Sparse high-dimensional linear regression. Estimating squared error and a phase transition
• Precise statistical analysis of classification accuracies for adversarial training
• The existence of maximum likelihood estimate in high-dimensional binary response generalized linear models
• Title not available (Why is that?)
• Tractability from overparametrization: the example of the negative perceptron
• Generalisation error in learning with random features and the hidden manifold model*
• Global and Simultaneous Hypothesis Testing for High-Dimensional Logistic Regression Models
• Large scale analysis of generalization error in learning using margin based classification methods
• A discussion of ``A note on universal inference by Tse and Davison
• A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers
• Large dimensional analysis of general margin based classification methods

This page was built for publication: The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2176606)