Component selection and smoothing in multivariate nonparametric regression
From MaRDI portal
(Redirected from Publication:869970)
Abstract: We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The ``COSSO is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO does model selection by applying a novel soft thresholding type operation to the function components. We give an equivalent formulation of the COSSO estimator which leads naturally to an iterative algorithm. We compare the COSSO with MARS, a popular method that builds functional ANOVA models, in simulations and real examples. The COSSO method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets. The COSSO gives very competitive performance in these studies.
Recommendations
- Component selection and smoothing for nonparametric regression in exponential families
- Selection of components and degrees of smoothing via Lasso in high dimensional nonparametric additive models
- Smoothing parameter selection for nonparametric regression using smoothing spline
- scientific article; zbMATH DE number 1036034
- Variable selection in nonparametric regression with continuous covariates
- Gradient-based smoothing parameter selection for nonparametric regression estimation
- Simultaneous selection of variables and smoothing parameters in structured additive regression models
- Variable selection in nonparametric functional concurrent regression
Cites work
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 3703820 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 472952 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A Statistical View of Some Chemometrics Regression Tools
- Adaptive Model Selection
- Atomic Decomposition by Basis Pursuit
- Benchmarking least squares support vector machine classifiers
- Better Subset Regression Using the Nonnegative Garrote
- Inference After Model Selection
- Least angle regression. (With discussion)
- Multivariate adaptive regression splines
- Natural spline functions, their associated eigenvalue problem
- On Measuring and Correcting the Effects of Data Mining and Model Selection
- Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
- Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
- Smoothing spline ANOVA models
- Structural modelling with sparse kernels
- Variable Selection and Model Building via Likelihood Basis Pursuit
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- Sparse estimation for functional semiparametric additive models
- Fixed and random effects selection in nonparametric additive mixed models
- High-dimensional local linear regression under sparsity and convex losses
- A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates
- A unified approach to error bounds for structured convex optimization problems
- High-dimensional Bayesian inference in nonparametric additive models
- Predictive functional linear models with diverging number of semiparametric single-index interactions
- Robust variable selection for the varying coefficient model based on composite \(L_1\)-\(L_2\) regression
- Modeling interactive components by coordinate kernel polynomial models
- Sparsistent and constansistent estimation of the varying-coefficient model with a diverging number of predictors
- Lasso in Infinite dimension: application to variable selection in functional multivariate linear regression
- A doubly sparse approach for group variable selection
- Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean
- Learning general sparse additive models from point queries in high dimensions
- Component selection and smoothing for nonparametric regression in exponential families
- Variable selection in additive models via hierarchical sparse penalty
- Functional ANOVA with multiple distributions: implications for the sensitivity analysis of computer experiments
- Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
- Functional Horseshoe Priors for Subspace Shrinkage
- High-dimensional additive modeling
- Automatic component selection in additive modeling of French national electricity load forecasting
- A screening approach for non-parametric global sensitivity analysis
- Regularization and model selection with categorical predictors and effect modifiers in generalized linear models
- Kernel meets sieve: post-regularization confidence bands for sparse additive model
- Kernel Knockoffs Selection for Nonparametric Additive Models
- Improved Estimation of High-dimensional Additive Models Using Subspace Learning
- Post-injection trapping of mobile \(\text{CO}_{2}\) in deep aquifers: assessing the importance of model and parameter uncertainties
- A necessary condition for the strong oracle property
- Error analysis for coefficient-based regularized regression in additive models
- Model selection for Cox models with time-varying coefficients
- Iterative smooth \(L_{1/2}\) algorithm for variable selection
- Reluctant generalized additive modeling
- Flexible and Interpretable Models for Survival Data
- Error variance estimation in ultrahigh-dimensional additive models
- Variable Selection Via Thompson Sampling
- Nonparametric variable screening for multivariate additive models
- Generalized Hoeffding-Sobol decomposition for dependent variables -- application to sensitivity analysis
- Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression
- Fast state-space methods for inferring dendritic synaptic connectivity
- Practical variable selection for generalized additive models
- Structured sparsity through convex optimization
- GRID: a variable selection and structure discovery method for high dimensional nonparametric regression
- Sparsity in multiple kernel learning
- Reproducing Kernel Hilbert Spaces for Penalized Regression: A Tutorial
- A Tweedie Compound Poisson Model in Reproducing Kernel Hilbert Space
- Testing the equality of linear single-index models
- Least angle and \(\ell _{1}\) penalized regression: a review
- Fast Bayesian model assessment for nonparametric additive regression
- Penalized likelihood and Bayesian function selection in regression models
- Bayesian variable selection in generalized additive partial linear models
- Variable selection for non-parametric quantile regression via smoothing spline analysis of variance
- Covariate selection for multilevel models with missing data
- Backfitting algorithms for total-variation and empirical-norm penalized additive modelling with high-dimensional data
- Sparse nonparametric regression with regularized tensor product kernel
- Additive model selection
- A new large-scale learning algorithm for generalized additive models
- Multiple-population shrinkage estimation via sliced inverse regression
- CLEAR: covariant least-square refitting with applications to image restoration
- Composite coefficient of determination and its application in ultrahigh dimensional variable screening
- A selective review of group selection in high-dimensional models
- Variable selection for nonparametric quantile regression via measurement error model
- Partially Linear Functional Additive Models for Multivariate Functional Data
- A group bridge approach for component selection in nonparametric accelerated failure time additive regression model
- Bayesian Model Selection in Additive Partial Linear Models Via Locally Adaptive Splines
- Variable selection in nonparametric additive models
- A simple approach for varying-coefficient model selection
- Kernel Ordinary Differential Equations
- Statistical inference in sparse high-dimensional additive models
- Variable selection based on squared derivative averages
- Estimating functions and derivatives via adaptive penalized splines
- Kernel continuum regression
- Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection
- Nonparametric Interaction Selection
- Nonparametric augmented probability weighting with sparsity
- Spike-and-slab priors for function selection in structured additive regression models
- Variable selection in robust semiparametric modeling for longitudinal data
- Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers
- A sparse additive model for high-dimensional interactions with an exposure variable
- Bayesian pathway selection
- Transposable regularized covariance models with an application to missing data imputation
- Nonparametric additive beta regression for fractional response with application to body fat data
- A reluctant additive model framework for interpretable nonlinear individualized treatment rules
- Input selection and shrinkage in multiresponse linear regression
- Kernel Cox partially linear regression: building predictive models for cancer patients' survival
- Joint semiparametric kernel network regression
- Block-wise primal-dual algorithms for large-scale doubly penalized ANOVA modeling
- Selecting the amount of smoothing in nonparametric regression estimation for complex surveys
- Improving the prediction performance of the Lasso by subtracting the additive structural noises
- Model selection in nonparametric hazard regression
- Feature selection algorithms in generalized additive models under concurvity
- Empirical Bayes oracle uncertainty quantification for regression
- Lag selection in stochastic additive models
- D-learning to estimate optimal individual treatment rules
- P-splines with an \(\ell_1\) penalty for repeated measures
- Learning non-parametric basis independent models from point queries via low-rank methods
- Additive monotone regression in high and lower dimensions
- Parametric component detection and variable selection in varying-coefficient partially linear models
- Metamodel construction for sensitivity analysis
- Layer-wise learning strategy for nonparametric tensor product smoothing spline regression and graphical models
- Variable selection in functional additive regression models
This page was built for publication: Component selection and smoothing in multivariate nonparametric regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q869970)
