Linear and convex aggregation of density estimators
From MaRDI portal
Publication:734530
DOI10.3103/S1066530707030052zbMATH Open1231.62057arXivmath/0605292OpenAlexW2101675718MaRDI QIDQ734530FDOQ734530
Philippe Rigollet, Alexandre B. Tsybakov
Publication date: 13 October 2009
Published in: Mathematical Methods of Statistics (Search for Journal in Brave)
Abstract: We study the problem of linear and convex aggregation of estimators of a density with respect to the mean squared risk. We provide procedures for linear and convex aggregation and we prove oracle inequalities for their risks. We also obtain lower bounds showing that these procedures are rate optimal in a minimax sense. As an example, we apply general results to aggregation of multivariate kernel density estimators with different bandwidths. We show that linear and convex aggregates mimic the kernel oracles in asymptotically exact sense for a large class of kernels including Gaussian, Silverman's and Pinsker's ones. We prove that, for Pinsker's kernel, the proposed aggregates are sharp asymptotically minimax simultaneously over a large scale of Sobolev classes of densities. Finally, we provide simulations demonstrating performance of the convex aggregation procedure.
Full work available at URL: https://arxiv.org/abs/math/0605292
Recommendations
- Learning Theory and Kernel Machines
- Optimal exponential bounds for aggregation of density estimators
- Optimal exponential bounds for aggregation of estimators for the Kullback-Leibler loss
- Generalized mirror averaging and \(D\)-convex aggregation
- A combined strategy for multivariate density estimation
Density estimation (62G07) Asymptotic properties of nonparametric inference (62G20) Estimation in multivariate analysis (62H12) Inequalities; stochastic orderings (60E15)
Cites Work
- Title not available (Why is that?)
- Exact mean integrated squared error
- Title not available (Why is that?)
- An asymptotically optimal window selection rule for kernel density estimates
- Title not available (Why is that?)
- Learning Theory and Kernel Machines
- Title not available (Why is that?)
- Mixing strategies for density estimation.
- Functional aggregation for nonparametric regression.
- Aggregation for Gaussian regression
- Model selection via testing: an alternative to (penalized) maximum likelihood estimators.
- On estimating a density using Hellinger distance and some other strange facts
- Optimal filtering of square-integrable signals in Gaussian noise
- Combinatorial methods in density estimation
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- Nonparametric estimation of smooth probability densities in \(L_ 2\)
- Optimal rates and constants in \(L_ 2\)-minimax estimation of probability density functions
- LAN in Problems of Nonparametric Estimation of Functions and Lower Bounds for Quadratic Risks
- Adaptive density estimation using the blockwise Stein method
- From \(\varepsilon\)-entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
- Quasi-universal bandwidth selection for kernel density estimators
Cited In (51)
- Aggregation of affine estimators
- Sharp oracle inequalities for aggregation of affine estimators
- Aggregation of estimators and stochastic optimization
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- On adaptive minimax density estimation on \(\mathbb R^d\)
- A new approach to estimator selection
- Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality
- Optimal Kullback-Leibler aggregation in mixture density estimation by maximum likelihood
- Oracle inequalities and adaptive estimation in the convolution structure density model
- Optimal model selection in density estimation
- A data-driven kernel estimator of the density function
- Performance of empirical risk minimization in linear aggregation
- Random average shifted histograms
- Bandwidth selection for kernel density estimation: a review of fully automatic selectors
- Fast adaptive estimation of log-additive exponential models in Kullback-Leibler divergence
- Sharp adaptive drift estimation for ergodic diffusions: the multivariate case
- Sharp oracle inequalities for low-complexity priors
- Optimal bounds for aggregation of affine estimators
- Adaptive estimation over anisotropic functional classes via oracle approach
- Optimal Kernel Selection for Density Estimation
- Density estimation with stagewise optimization of the empirical risk
- A general procedure to combine estimators
- Spatial aggregation of local likelihood estimates with applications to classification
- Generalized mirror averaging and \(D\)-convex aggregation
- SPADES and mixture models
- Title not available (Why is that?)
- Concentration inequalities for the exponential weighting method
- A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
- Oracle inequalities and upper bounds for kernel density estimators on manifolds and more general metric spaces
- Aggregation of spectral density estimators
- Title not available (Why is that?)
- Estimator selection in the Gaussian setting
- A unified scheme for generalizing cardinality estimators to sum aggregation
- Stacked grenander and rearrangement estimators of a discrete distribution
- Bagging of density estimators
- Better than the best? Answers via model ensemble in density-based clustering
- Averaging of density kernel estimators
- Model selection for density estimation with \(\mathbb L_2\)-loss
- Kullback-Leibler aggregation and misspecified generalized linear models
- Multivariate density estimation under sup-norm loss: oracle approach, adaptation and independence structure
- Sampling from non-smooth distributions through Langevin diffusion
- Nonparametric estimation of the anisotropic probability density of mixed variables
- The two-sample problem for Poisson processes: adaptive tests with a nonasymptotic wild bootstrap approach
- Ordered smoothers with exponential weighting
- PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting
- Sparse estimation by exponential weighting
- Estimator selection with respect to Hellinger-type risks
- Statistical estimation with model selection
- Aggregating estimates by convex optimization
- On aggregation of uncensored and censored observations
- A combined strategy for multivariate density estimation
Uses Software
This page was built for publication: Linear and convex aggregation of density estimators
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q734530)
