Risk bounds for model selection via penalization
From MaRDI portal
Publication:1291160
The authors develop performance bounds for criteria of model selection, using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an added penalty term roughly proportional to the number of parameters needed to describe the model divided by the number of observations. Most of the presented examples involve density or regression estimation settings, and the authors focus on the problem of estimating the unknown density or regression function. It is shown that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve. The connection between model selection via penalization and adaptation in the minimax sense is pointed out. Such illustrations of the introduced method as penalized maximum likelihood, projection or least squares estimation are provided. The models involve commonly used finite dimensional expansions such as piecewise polynomials with fixed or variable knots, trigonometric polynomials, wavelets, neural nets, and related nonlinear expansions defined by superposition of ridge functions.
Recommendations
- scientific article; zbMATH DE number 17222
- Minimal penalties for Gaussian model selection
- scientific article; zbMATH DE number 1321826
- Model selection criteria based on computationally intensive estimators of the expected optimism
- scientific article; zbMATH DE number 1064642
- Model selection: a Lagrange optimization approach
- scientific article; zbMATH DE number 2212144
- Gaussian model selection
Cited in
(only showing first 100 items - show all)- Mixing least-squares estimators when the variance is unknown
- Evaluation and selection of models for out-of-sample prediction when the sample size is small relative to the complexity of the data-generating process
- Nonparametric adaptive estimation for integrated diffusions
- On the stability of the risk hull method for projection estimators
- Empirical Bayesian test of the smoothness
- Gaussian model selection with an unknown variance
- An improved model averaging scheme for logistic regression
- Detection of multiple changes in a sequence of dependent variables
- Rates of convergence of estimates, Kolmogorov's entropy and the dimensionality reduction principle in regression
- Image denoising: Pointwise adaptive approach
- Model selection in nonparametric regression
- Nonlinear orthogonal series estimates for random design regression
- Complexity penalized support estimation.
- Compound decision theory and empirical Bayes methods
- Estimating the intensity of a random measure by histogram type estimators
- Penalized projection estimators of the Aalen multiplicative intensity
- Information-theoretic determination of minimax rates of convergence
- Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression
- Combining different procedures for adaptive regression
- Smooth discrimination analysis
- Model selection for regression on a fixed design
- Lower bounds for the rate of convergence in nonparametric pattern recognition
- Oracle inequalities for the stochastic differential equations
- Estimator selection: a new method with applications to kernel density estimation
- A new approach to estimator selection
- Minimax-rate adaptive nonparametric regression with unknown correlations of errors
- Sobolev-Hermite versus Sobolev nonparametric density estimation on \(\mathbb{R}\)
- Fast adaptive estimation of log-additive exponential models in Kullback-Leibler divergence
- Non-parametric estimation of the diffusion coefficient from noisy data
- Model selection for Gaussian regression with random design
- Adaptive estimation of mean and volatility functions in (auto-)regressive models.
- A new algorithm for fixed design regression and denoising
- Asymptotically exact minimax estimation in sup-norm for anisotropic Hölder classes
- On a method of empirical risk minimization
- New dependence coefficients. Examples and applications to statistics
- General empirical Bayes wavelet methods and exactly adaptive minimax estimation
- Mixing strategies for density estimation.
- Adaptive estimation of a quadratic functional by model selection.
- Rates of convergence of posterior distributions.
- Adaptive estimation in autoregression or \(\beta\)-mixing regression via model selection
- Sharp adaptive estimation of linear functionals.
- Random rates in anisotropic regression. (With discussion)
- M-estimation using penalties or sieves
- Free knot splines in concave extended linear modeling
- Consistent covariate selection and post model selection inference in semiparametric regression.
- Multiscale likelihood analysis and complexity penalized estimation.
- Convergence rates for posterior distributions and adaptive estimation
- Efficient robust nonparametric estimation in a semimartingale regression model
- Adaptive estimation of linear functionals in functional linear models
- Sharp oracle inequalities for aggregation of affine estimators
- Adaptive estimation of the conditional intensity of marker-dependent counting processes
- Honest adaptive confidence bands and self-similar functions
- Oracle inequalities for cross-validation type procedures
- Selecting the length of a principal curve within a Gaussian model
- Model selection in regression under structural constraints
- Optimal model selection in heteroscedastic regression using piecewise polynomial functions
- Adaptive complexity regularization for linear inverse problems
- Adaptive estimation of linear functionals by model selection
- Simultaneous estimation of the mean and the variance in heteroscedastic Gaussian regression
- Cumulative distribution function estimation under interval censoring case 1
- General oracle inequalities for model selection
- Model selection by resampling penalization
- Dimension reduction and variable selection in case control studies via regularized likelihood optimization
- Plugin procedure in segmentation and application to hyperspectral image segmentation
- The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso)
- Inhomogeneous and anisotropic conditional density estimation from dependent data
- Model selection and sharp asymptotic minimaxity
- Adaptive kernel methods using the balancing principle
- The Goldenshluger-Lepski method for constrained least-squares estimators over RKHSs
- Nonparametric estimation of jump rates for a specific class of piecewise deterministic Markov processes
- Approximate \(\ell_0\)-penalized estimation of piecewise-constant signals on graphs
- Asymptotic analysis of model selection criteria for general hidden Markov models
- Optimal estimation of variance in nonparametric regression with random design
- Drift estimation on non compact support for diffusion models
- Model selection for the robust efficient signal processing observed with small Lévy noise
- Estimating a density, a hazard rate, and a transition intensity via the \(\rho\)-estimation method
- Oracle posterior contraction rates under hierarchical priors
- A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
- Minimax bounds for Besov classes in density estimation
- Estimates for the SVD of the truncated Fourier transform on \(L^2(\cosh (B|\cdot |))\) and stable analytic continuation
- Set structured global empirical risk minimizers are rate optimal in general dimensions
- Graph signal denoising using \(t\)-shrinkage priors
- On robust learning in the canonical change point problem under heavy tailed errors in finite and growing dimensions
- Empirical variance minimization with applications in variance reduction and optimal control
- Adaptive density estimation on bounded domains under mixing conditions
- On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising
- Model selection: from theory to practice
- A general framework for Bayes structured linear models
- A look at robustness and stability of \(\ell_1\)-versus \(\ell_0\)-regularization: discussion of papers by Bertsimas et al. and Hastie et al.
- Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection
- Maxisets for model selection
- Non parametric estimation of the diffusion coefficients of a diffusion with jumps
- Regression function estimation on non compact support in an heteroscesdastic model
- Bayesian linear size-and-shape regression with applications to face data
- Non-parametric Poisson regression from independent and weakly dependent observations by model selection
- Adaptive procedure for Fourier estimators: application to deconvolution and decompounding
- Nonparametric intensity estimation from noisy observations of a Poisson process under unknown error distribution
- Testing the regularity of a smooth signal
- Estimation and model selection for model-based clustering with the conditional classification likelihood
- Adaptive estimation over anisotropic functional classes via oracle approach
This page was built for publication: Risk bounds for model selection via penalization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1291160)
