Component selection and smoothing in multivariate nonparametric regression
From MaRDI portal
Publication:869970
DOI10.1214/009053606000000722zbMATH Open1106.62041arXivmath/0702659OpenAlexW2080726496MaRDI QIDQ869970FDOQ869970
Authors: Yi Lin, Hao Helen Zhang 
Publication date: 12 March 2007
Published in: The Annals of Statistics (Search for Journal in Brave)
Abstract: We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The ``COSSO is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO does model selection by applying a novel soft thresholding type operation to the function components. We give an equivalent formulation of the COSSO estimator which leads naturally to an iterative algorithm. We compare the COSSO with MARS, a popular method that builds functional ANOVA models, in simulations and real examples. The COSSO method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets. The COSSO gives very competitive performance in these studies.
Full work available at URL: https://arxiv.org/abs/math/0702659
Recommendations
- Component selection and smoothing for nonparametric regression in exponential families
- Selection of components and degrees of smoothing via Lasso in high dimensional nonparametric additive models
- Smoothing parameter selection for nonparametric regression using smoothing spline
- scientific article; zbMATH DE number 1036034
- Variable selection in nonparametric regression with continuous covariates
- Gradient-based smoothing parameter selection for nonparametric regression estimation
- Simultaneous selection of variables and smoothing parameters in structured additive regression models
- Variable selection in nonparametric functional concurrent regression
Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Analysis of variance and covariance (ANOVA) (62J10)
Cites Work
- Smoothing spline ANOVA models
- Least angle regression. (With discussion)
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Title not available (Why is that?)
- Multivariate adaptive regression splines
- Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
- Atomic Decomposition by Basis Pursuit
- Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Better Subset Regression Using the Nonnegative Garrote
- On Measuring and Correcting the Effects of Data Mining and Model Selection
- A Statistical View of Some Chemometrics Regression Tools
- Variable Selection and Model Building via Likelihood Basis Pursuit
- Structural modelling with sparse kernels
- Benchmarking least squares support vector machine classifiers
- Title not available (Why is that?)
- Title not available (Why is that?)
- Adaptive Model Selection
- Natural spline functions, their associated eigenvalue problem
- Inference After Model Selection
Cited In (only showing first 100 items - show all)
- Statistical inference in sparse high-dimensional additive models
- Spike-and-slab priors for function selection in structured additive regression models
- Kernel continuum regression
- A sparse additive model for high-dimensional interactions with an exposure variable
- Bayesian pathway selection
- Variable selection in robust semiparametric modeling for longitudinal data
- Selecting the amount of smoothing in nonparametric regression estimation for complex surveys
- Transposable regularized covariance models with an application to missing data imputation
- Model selection in nonparametric hazard regression
- Input selection and shrinkage in multiresponse linear regression
- Layer-wise learning strategy for nonparametric tensor product smoothing spline regression and graphical models
- Learning non-parametric basis independent models from point queries via low-rank methods
- Parametric component detection and variable selection in varying-coefficient partially linear models
- An analysis of penalized interaction models
- Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood
- Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space
- Multiresolution functional ANOVA for large-scale, many-input computer experiments
- Using recursive algorithms for the efficient identification of smoothing spline ANOVA models
- Symmetrical and non-symmetrical variants of three-way correspondence analysis for ordered variables
- Nonparametric variable selection and its application to additive models
- Semiparametric regression models with additive nonparametric components and high dimensional parametric components
- A hierarchical multiple kernel support vector machine for customer churn prediction using longitudinal behavioral data
- Additive models with trend filtering
- Testing for additivity in non-parametric regression
- Learning sparse gradients for variable selection and dimension reduction
- Robust penalized logistic regression with truncated loss functions
- Robust spline-based variable selection in varying coefficient model
- Functional index coefficient models with variable selection
- Feature selection in the Laplacian support vector machine
- Elicitation of multiattribute value functions through high dimensional model representations: monotonicity and interactions
- Component selection in the additive regression model
- Another look at linear programming for feature selection via methods of regularization
- Bayesian group Lasso for nonparametric varying-coefficient models with application to functional genome-wide association studies
- A fused Lasso approach to nonstationary spatial covariance estimation
- Penalized variable selection procedure for Cox models with semiparametric relative risk
- Refined generalization bounds of gradient learning over reproducing kernel Hilbert spaces
- Sparse optimization for nonconvex group penalized estimation
- Structured estimation for the nonparametric Cox model
- Bias-corrected inference for multivariate nonparametric regression: model selection and oracle property
- Fixed and random effects selection in nonparametric additive mixed models
- Robust variable selection for the varying coefficient model based on composite \(L_1\)-\(L_2\) regression
- A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates
- A unified approach to error bounds for structured convex optimization problems
- Lasso in Infinite dimension: application to variable selection in functional multivariate linear regression
- High-dimensional Bayesian inference in nonparametric additive models
- Component selection and smoothing for nonparametric regression in exponential families
- Learning general sparse additive models from point queries in high dimensions
- Functional Horseshoe Priors for Subspace Shrinkage
- High-dimensional additive modeling
- Regularization and model selection with categorical predictors and effect modifiers in generalized linear models
- Kernel Knockoffs Selection for Nonparametric Additive Models
- Error variance estimation in ultrahigh-dimensional additive models
- Reluctant generalized additive modeling
- Flexible and Interpretable Models for Survival Data
- Structured sparsity through convex optimization
- Generalized Hoeffding-Sobol decomposition for dependent variables -- application to sensitivity analysis
- Practical variable selection for generalized additive models
- Sparsity in multiple kernel learning
- Testing the equality of linear single-index models
- Least angle and \(\ell _{1}\) penalized regression: a review
- Fast Bayesian model assessment for nonparametric additive regression
- Penalized likelihood and Bayesian function selection in regression models
- A selective review of group selection in high-dimensional models
- Additive model selection
- Multiple-population shrinkage estimation via sliced inverse regression
- Partially Linear Functional Additive Models for Multivariate Functional Data
- Kernel Ordinary Differential Equations
- Variable selection in nonparametric additive models
- A simple approach for varying-coefficient model selection
- Estimating functions and derivatives via adaptive penalized splines
- Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection
- Nonparametric augmented probability weighting with sparsity
- Nonparametric Interaction Selection
- Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers
- A reluctant additive model framework for interpretable nonlinear individualized treatment rules
- Kernel Cox partially linear regression: building predictive models for cancer patients' survival
- Joint semiparametric kernel network regression
- Block-wise primal-dual algorithms for large-scale doubly penalized ANOVA modeling
- Nonparametric additive beta regression for fractional response with application to body fat data
- Feature selection algorithms in generalized additive models under concurvity
- Improving the prediction performance of the Lasso by subtracting the additive structural noises
- Lag selection in stochastic additive models
- Empirical Bayes oracle uncertainty quantification for regression
- D-learning to estimate optimal individual treatment rules
- P-splines with an \(\ell_1\) penalty for repeated measures
- Metamodel construction for sensitivity analysis
- Additive monotone regression in high and lower dimensions
- Variable selection in functional additive regression models
- Estimation of sparse functional additive models with adaptive group Lasso
- Domain selection for the varying coefficient model via local polynomial regression
- Partially linear structure identification in generalized additive models with NP-dimensionality
- Regularization and model selection for quantile varying coefficient model with categorical effect modifiers
- Bayesian neural networks for selection of drug sensitive genes
- Saturating splines and feature selection
- Kernel variable selection for multicategory support vector machines
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- A distance metric-based space-filling subsampling method for nonparametric models
- Grouped variable selection with discrete optimization: computational and statistical perspectives
- HARFE: hard-ridge random feature expansion
- Efficient kernel-based variable selection with sparsistency
Uses Software
This page was built for publication: Component selection and smoothing in multivariate nonparametric regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q869970)
