Component selection and smoothing in multivariate nonparametric regression
From MaRDI portal
Abstract: We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The ``COSSO is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO does model selection by applying a novel soft thresholding type operation to the function components. We give an equivalent formulation of the COSSO estimator which leads naturally to an iterative algorithm. We compare the COSSO with MARS, a popular method that builds functional ANOVA models, in simulations and real examples. The COSSO method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets. The COSSO gives very competitive performance in these studies.
Recommendations
- Component selection and smoothing for nonparametric regression in exponential families
- Selection of components and degrees of smoothing via Lasso in high dimensional nonparametric additive models
- Smoothing parameter selection for nonparametric regression using smoothing spline
- scientific article; zbMATH DE number 1036034
- Variable selection in nonparametric regression with continuous covariates
- Gradient-based smoothing parameter selection for nonparametric regression estimation
- Simultaneous selection of variables and smoothing parameters in structured additive regression models
- Variable selection in nonparametric functional concurrent regression
Cites work
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 3703820 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 472952 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A Statistical View of Some Chemometrics Regression Tools
- Adaptive Model Selection
- Atomic Decomposition by Basis Pursuit
- Benchmarking least squares support vector machine classifiers
- Better Subset Regression Using the Nonnegative Garrote
- Inference After Model Selection
- Least angle regression. (With discussion)
- Multivariate adaptive regression splines
- Natural spline functions, their associated eigenvalue problem
- On Measuring and Correcting the Effects of Data Mining and Model Selection
- Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
- Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
- Smoothing spline ANOVA models
- Structural modelling with sparse kernels
- Variable Selection and Model Building via Likelihood Basis Pursuit
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- Component selection in the additive regression model
- Fast Bayesian model assessment for nonparametric additive regression
- A sparse additive model for high-dimensional interactions with an exposure variable
- Nonparametric variable selection and its application to additive models
- Selecting the amount of smoothing in nonparametric regression estimation for complex surveys
- Functional Horseshoe Priors for Subspace Shrinkage
- Semiparametric regression models with additive nonparametric components and high dimensional parametric components
- Bayesian pathway selection
- Robust variable selection for the varying coefficient model based on composite \(L_1\)-\(L_2\) regression
- High-dimensional additive modeling
- Practical variable selection for generalized additive models
- An analysis of penalized interaction models
- Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood
- Transposable regularized covariance models with an application to missing data imputation
- Partially Linear Functional Additive Models for Multivariate Functional Data
- Robust penalized logistic regression with truncated loss functions
- Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space
- A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates
- Statistical inference in sparse high-dimensional additive models
- Bias-corrected inference for multivariate nonparametric regression: model selection and oracle property
- Variable selection in nonparametric additive models
- Reluctant generalized additive modeling
- Flexible and Interpretable Models for Survival Data
- Using recursive algorithms for the efficient identification of smoothing spline ANOVA models
- Learning non-parametric basis independent models from point queries via low-rank methods
- Variable selection in robust semiparametric modeling for longitudinal data
- Robust spline-based variable selection in varying coefficient model
- Penalized likelihood and Bayesian function selection in regression models
- A simple approach for varying-coefficient model selection
- A unified approach to error bounds for structured convex optimization problems
- Regularization and model selection with categorical predictors and effect modifiers in generalized linear models
- Another look at linear programming for feature selection via methods of regularization
- Bayesian group Lasso for nonparametric varying-coefficient models with application to functional genome-wide association studies
- Parametric component detection and variable selection in varying-coefficient partially linear models
- A hierarchical multiple kernel support vector machine for customer churn prediction using longitudinal behavioral data
- A selective review of group selection in high-dimensional models
- Spike-and-slab priors for function selection in structured additive regression models
- Kernel continuum regression
- Structured sparsity through convex optimization
- Sparse optimization for nonconvex group penalized estimation
- Lasso in Infinite dimension: application to variable selection in functional multivariate linear regression
- Generalized Hoeffding-Sobol decomposition for dependent variables -- application to sensitivity analysis
- Structured estimation for the nonparametric Cox model
- Penalized variable selection procedure for Cox models with semiparametric relative risk
- Model selection in nonparametric hazard regression
- Fixed and random effects selection in nonparametric additive mixed models
- Estimating functions and derivatives via adaptive penalized splines
- Component selection and smoothing for nonparametric regression in exponential families
- Testing for additivity in non-parametric regression
- Input selection and shrinkage in multiresponse linear regression
- Multiresolution functional ANOVA for large-scale, many-input computer experiments
- Additive model selection
- Sparsity in multiple kernel learning
- Functional index coefficient models with variable selection
- Elicitation of multiattribute value functions through high dimensional model representations: monotonicity and interactions
- Error variance estimation in ultrahigh-dimensional additive models
- Multiple-population shrinkage estimation via sliced inverse regression
- A fused Lasso approach to nonstationary spatial covariance estimation
- Testing the equality of linear single-index models
- Additive models with trend filtering
- Layer-wise learning strategy for nonparametric tensor product smoothing spline regression and graphical models
- Symmetrical and non-symmetrical variants of three-way correspondence analysis for ordered variables
- Least angle and \(\ell _{1}\) penalized regression: a review
- High-dimensional Bayesian inference in nonparametric additive models
- Feature selection in the Laplacian support vector machine
- Learning general sparse additive models from point queries in high dimensions
- Refined generalization bounds of gradient learning over reproducing kernel Hilbert spaces
- Learning sparse gradients for variable selection and dimension reduction
- Kernel Ordinary Differential Equations
- Kernel Knockoffs Selection for Nonparametric Additive Models
- Modal additive models with data-driven structure identification
- Efficient functional Lasso kernel smoothing for high-dimensional additive regression
- Convex Bidirectional Large Margin Classifiers
- An Outer-Product-of-Gradient Approach to Dimension Reduction and its Application to Classification in High Dimensional Space
- Data-adaptive additive modeling
- Modeling interactive components by coordinate kernel polynomial models
- Improving the prediction performance of the Lasso by subtracting the additive structural noises
- A General Framework of Nonparametric Feature Selection in High-Dimensional Data
- Bayesian variable selection via a benchmark in normal linear models
- Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection
- Automatic structure recovery for generalized additive models
- Adaptive penalized splines for data smoothing
- Integrative weighted group Lasso and generalized local quadratic approximation
- Learning sparse conditional distribution: an efficient kernel-based approach
- Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression
- Nonparametric augmented probability weighting with sparsity
- Measures of Uncertainty for Shrinkage Model Selection
- Additive monotone regression in high and lower dimensions
- Time-varying quantile single-index model for multivariate responses
- Nonparametric additive beta regression for fractional response with application to body fat data
- Post-injection trapping of mobile \(\text{CO}_{2}\) in deep aquifers: assessing the importance of model and parameter uncertainties
- Error analysis for coefficient-based regularized regression in additive models
- CLEAR: covariant least-square refitting with applications to image restoration
- Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers
- Variable selection in functional additive regression models
- Estimation of sparse functional additive models with adaptive group Lasso
- Nonparametric Interaction Selection
- High-dimensional local linear regression under sparsity and convex losses
- GRID: a variable selection and structure discovery method for high dimensional nonparametric regression
- Least angle regression for model selection
This page was built for publication: Component selection and smoothing in multivariate nonparametric regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q869970)
