Estimating a regression function in exponential families by model selection

DOI10.3150/23-BEJ1649MaRDI QIDQ6201869zbMATH OpenOpenAlexWikidataFDO

Authors J. T. Chen

Publication date 26 March 2024

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2203.06656, https://projecteuclid.org/journals/bernoulli/volume-30/issue-2/Estimating-a-regression-function-in-exponential-families-by-model-selection/10.3150/23-BEJ1649.full

zbMATH Keywords

model selectionvariable selectionReLU neural networksgeneralized additive structuremultiple index structureregression in exponential family

Mathematics Subject Classification ID

Statistics (62-XX) Biology and other natural sciences (92-XX)

Abstract: Let $X_1=(W_1,Y_1),ldots,X_n=(W_n,Y_n)$ be $n$ pairs of independent random variables. We assume that, for each $iin1,ldots,n$, the conditional distribution of $Y_i$ given $W_i$ belongs to a one-parameter exponential family with parameter $$, or at least, is close enough to a distribution of this form. The objective of the present paper is to estimate these conditional distributions on the basis of the observation $$ and to do so, we propose a model selection procedure together with a non-asymptotic risk bound for the resulted estimator with respect to a Hellinger-type distance. When $$ does exist, the procedure allows to obtain an estimator $$ of $$ adapted to a wide range of the anisotropic Besov spaces. When $$ has a general additive or multiple index structure, we construct suitable models and show the resulted estimators by our procedure based on such models can circumvent the curse of dimensionality. Moreover, we consider model selection problems for ReLU neural networks and provide an example where estimation based on neural networks enjoys a much faster converge rate than the classical models. Finally, we apply this procedure to solve variable selection problem in exponential families. The proofs in the paper rely on bounding the VC dimensions of several collections of functions, which can be of independent interest.

Cites work

• scientific article; zbMATH DE number 3703310 (Why is no real title available?)
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• A multiscale variance stabilization for binomial sequence proportion estimation
• A new method for estimation and model selection: \(\rho\)-estimation
• Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection
• Deep learning
• Estimating composite functions by model selection
• More on the estimation of distribution densities
• Multidimensional Spline Approximation
• Multiscale generalised linear models for nonparametric function estimation
• Non-parametric Poisson regression from independent and weakly dependent observations by model selection
• Nonlinear approximation and (deep) ReLU networks
• Nonparametric Estimation of a Regression Function that is Smooth in a Domain in $R^k$
• Nonparametric regression in exponential families
• Nonparametric regression using deep neural networks with ReLU activation function
• Optimal global rates of convergence for nonparametric regression
• Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
• Rho-estimators revisited: general theory and applications
• Sparse Poisson regression with penalized weighted score function
• Theory of function spaces. III
• Wavelet characterizations for anisotropic Besov spaces
• Wavelet shrinkage for natural exponential families with quadratic variance functions
• Weak convergence and empirical processes. With applications to statistics
• Weierstrass' function and chaos

Cited in

(1)

• Robust nonparametric regression based on deep ReLU neural networks

This page was built for publication: Estimating a regression function in exponential families by model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6201869)