Approximation of conditional densities by smooth mixtures of regressions
From MaRDI portal
Publication:973883
DOI10.1214/09-AOS765zbMATH Open1189.62060arXiv1010.0581OpenAlexW1969907345MaRDI QIDQ973883FDOQ973883
Publication date: 26 May 2010
Published in: The Annals of Statistics (Search for Journal in Brave)
Abstract: This paper shows that large nonparametric classes of conditional multivariate densities can be approximated in the Kullback--Leibler distance by different specifications of finite mixtures of normal regressions in which normal means and variances and mixing probabilities can depend on variables in the conditioning set (covariates). These models are a special case of models known as "mixtures of experts" in statistics and computer science literature. Flexible specifications include models in which only mixing probabilities, modeled by multinomial logit, depend on the covariates and, in the univariate case, models in which only means of the mixed normals depend flexibly on the covariates. Modeling the variance of the mixed normals by flexible functions of the covariates can weaken restrictions on the class of the approximable densities. Obtained results can be generalized to mixtures of general location scale densities. Rates of convergence and easy to interpret bounds are also obtained for different model specifications. These approximation results can be useful for proving consistency of Bayesian and maximum likelihood density estimators based on these models. The results also have interesting implications for applied researchers.
Full work available at URL: https://arxiv.org/abs/1010.0581
Recommendations
- Smooth density estimation with moment constraints using mixture distributions
- Regression density estimation using smooth adaptive Gaussian mixtures
- Smooth estimates of normal mixtures
- A Smooth Nonparametric Estimate of a Mixing Distribution Using Mixtures of Gaussians
- Maximum smoothed likelihood component density estimation in mixture models with known mixing proportions
mixtures of expertsBayesian conditional density estimationfinite mixtures of normal distributionssmoothly mixing regressions
Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Artificial intelligence (68T99) Approximation by other special function classes (41A30)
Cites Work
- Finite mixture models
- Regression density estimation using smooth adaptive Gaussian mixtures
- The Calculation of Posterior Distributions by Data Augmentation
- Practical Bayesian Density Estimation Using Mixtures of Normals
- Bayesian Analysis of Binary and Polychotomous Response Data
- Smoothly mixing regressions
- Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation
- A mixture likelihood approach for generalized linear models
- Estimating Mixtures of Normal Distributions and Switching Regressions
- Bayesian Inference in Mixtures-of-Experts and Hierarchical Mixtures-of-Experts Models With an Application to Speech Recognition
- Discrete Parameter Variation: Efficient Estimation of a Switching Regression Model
- Title not available (Why is that?)
- Bayesian nonparametrics
- Conditional Choice Probabilities and the Estimation of Dynamic Models
- Title not available (Why is that?)
- POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES
- Bayesian mixture of splines for spatially adaptive nonparametric regression
- Title not available (Why is that?)
- Error bounds for functional approximation and estimation using mixtures of experts
Cited In (25)
- ADAPTIVE BAYESIAN ESTIMATION OF CONDITIONAL DENSITIES
- Data science, big data and statistics
- On convergence rates of mixtures of polynomial experts
- Relabelling algorithms for mixture models with applications for large data sets
- A Universal Approximation Theorem for Mixture-of-Experts Models
- Bayesian Spectral Modeling for Multiple Time Series
- Dynamic network data envelopment analysis with a sequential structure and behavioural-causal analysis: application to the Chinese banking industry
- Adaptive Bayesian estimation of conditional discrete-continuous distributions with an application to stock market trading activity
- Evaluating consumers' choices of Medicare Part D plans: a study in behavioral welfare economics
- Density Forecasts in Panel Data Models: A Semiparametric Bayesian Perspective
- Bayesian artificial neural networks for frontier efficiency analysis
- Covariate-Assisted Bayesian Graph Learning for Heterogeneous Data
- POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES
- Maximum likelihood estimation in a semicontinuous survival model with covariates subject to detection limits
- On the aggregation of probability assessments: regularized mixtures of predictive densities for eurozone inflation and real interest rates
- Posterior consistency in conditional distribution estimation
- Smoothly mixing regressions
- Bayesian mixture modeling for spectral density estimation
- Bayesian regression with heteroscedastic error density and parametric mean function
- Incorporating causal modeling into data envelopment analysis for performance evaluation
- A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models
- Approximations of conditional probability density functions in Lebesgue spaces via mixture of experts models
- Bayesian regression with nonparametric heteroskedasticity
- Bayesian modeling of joint and conditional distributions
- Generalized smooth finite mixtures
Uses Software
This page was built for publication: Approximation of conditional densities by smooth mixtures of regressions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q973883)
