Component selection and smoothing in multivariate nonparametric regression
From MaRDI portal
(Redirected from Publication:869970)
Abstract: We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The ``COSSO is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO does model selection by applying a novel soft thresholding type operation to the function components. We give an equivalent formulation of the COSSO estimator which leads naturally to an iterative algorithm. We compare the COSSO with MARS, a popular method that builds functional ANOVA models, in simulations and real examples. The COSSO method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets. The COSSO gives very competitive performance in these studies.
Recommendations
- Component selection and smoothing for nonparametric regression in exponential families
- Selection of components and degrees of smoothing via Lasso in high dimensional nonparametric additive models
- Smoothing parameter selection for nonparametric regression using smoothing spline
- scientific article; zbMATH DE number 1036034
- Variable selection in nonparametric regression with continuous covariates
- Gradient-based smoothing parameter selection for nonparametric regression estimation
- Simultaneous selection of variables and smoothing parameters in structured additive regression models
- Variable selection in nonparametric functional concurrent regression
Cites work
- A Statistical View of Some Chemometrics Regression Tools
- Adaptive Model Selection
- Atomic Decomposition by Basis Pursuit
- Benchmarking least squares support vector machine classifiers
- Better Subset Regression Using the Nonnegative Garrote
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 3703820 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 472952 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Inference After Model Selection
- Least angle regression. (With discussion)
- Multivariate adaptive regression splines
- Natural spline functions, their associated eigenvalue problem
- On Measuring and Correcting the Effects of Data Mining and Model Selection
- Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
- Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
- Smoothing spline ANOVA models
- Structural modelling with sparse kernels
- Variable Selection and Model Building via Likelihood Basis Pursuit
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- A simple approach for varying-coefficient model selection
- Input selection and shrinkage in multiresponse linear regression
- High-dimensional additive modeling
- Kernel Knockoffs Selection for Nonparametric Additive Models
- Reluctant generalized additive modeling
- Flexible and Interpretable Models for Survival Data
- Multiresolution functional ANOVA for large-scale, many-input computer experiments
- Kernel continuum regression
- P-splines with an \(\ell_1\) penalty for repeated measures
- D-learning to estimate optimal individual treatment rules
- Penalized likelihood and Bayesian function selection in regression models
- Fast Bayesian model assessment for nonparametric additive regression
- Regularization and model selection for quantile varying coefficient model with categorical effect modifiers
- Partially linear structure identification in generalized additive models with NP-dimensionality
- Domain selection for the varying coefficient model via local polynomial regression
- Adaptive penalized splines for data smoothing
- Integrative weighted group Lasso and generalized local quadratic approximation
- Time-varying quantile single-index model for multivariate responses
- Combining meta-modeling and categorical indicators for global sensitivity analysis of long-running flow simulators with spatially dependent inputs
- A unified approach to error bounds for structured convex optimization problems
- A doubly sparse approach for group variable selection
- Post-injection trapping of mobile \(\text{CO}_{2}\) in deep aquifers: assessing the importance of model and parameter uncertainties
- Error analysis for coefficient-based regularized regression in additive models
- Fast state-space methods for inferring dendritic synaptic connectivity
- Improving the prediction performance of the Lasso by subtracting the additive structural noises
- Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space
- Sparse estimation for functional semiparametric additive models
- An RKHS-based approach to double-penalized regression in high-dimensional partially linear models
- Fixed and random effects selection in nonparametric additive mixed models
- A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates
- Generalized Hoeffding-Sobol decomposition for dependent variables -- application to sensitivity analysis
- Empirical Bayes oracle uncertainty quantification for regression
- Additive monotone regression in high and lower dimensions
- Using recursive algorithms for the efficient identification of smoothing spline ANOVA models
- Learning general sparse additive models from point queries in high dimensions
- Block-based refitting in \(\ell_{12}\) sparse regularization
- Learning sparse conditional distribution: an efficient kernel-based approach
- Modeling interactive components by coordinate kernel polynomial models
- Interpretable machine learning: fundamental principles and 10 grand challenges
- Symmetrical and non-symmetrical variants of three-way correspondence analysis for ordered variables
- Nonparametric variable screening for multivariate additive models
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- Predictive functional linear models with diverging number of semiparametric single-index interactions
- Nonparametric variable selection and its application to additive models
- GRID: a variable selection and structure discovery method for high dimensional nonparametric regression
- Kernel variable selection for multicategory support vector machines
- A hierarchical multiple kernel support vector machine for customer churn prediction using longitudinal behavioral data
- Robust spline-based variable selection in varying coefficient model
- Additive models with trend filtering
- Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean
- Doubly penalized estimation in additive regression with high-dimensional data
- Nonparametric additive beta regression for fractional response with application to body fat data
- Structured estimation for the nonparametric Cox model
- Variable selection in functional additive regression models
- Variable selection based on squared derivative averages
- Modal additive models with data-driven structure identification
- A necessary condition for the strong oracle property
- Iterative smooth \(L_{1/2}\) algorithm for variable selection
- Local independence feature screening for nonparametric and semiparametric models by marginal empirical likelihood
- An analysis of penalized interaction models
- Sparsistent and constansistent estimation of the varying-coefficient model with a diverging number of predictors
- Component selection in the additive regression model
- Model selection for Cox models with time-varying coefficients
- Robust penalized logistic regression with truncated loss functions
- Elicitation of multiattribute value functions through high dimensional model representations: monotonicity and interactions
- A fused Lasso approach to nonstationary spatial covariance estimation
- Component selection and smoothing for nonparametric regression in exponential families
- Variable selection in robust semiparametric modeling for longitudinal data
- Semiparametric regression models with additive nonparametric components and high dimensional parametric components
- Learning sparse gradients for variable selection and dimension reduction
- Parametric component detection and variable selection in varying-coefficient partially linear models
- Feature selection in the Laplacian support vector machine
- Saturating splines and feature selection
- Bayesian neural networks for selection of drug sensitive genes
- Metamodel construction for sensitivity analysis
- Functional ANOVA with multiple distributions: implications for the sensitivity analysis of computer experiments
- Error variance estimation in ultrahigh-dimensional additive models
- High-dimensional Bayesian inference in nonparametric additive models
- Spike-and-slab priors for function selection in structured additive regression models
- Structured kernel quantile regression
- Regularization and model selection with categorical predictors and effect modifiers in generalized linear models
- Univariate measurement error selection likelihood for variable selection of additive model
- Efficient kernel-based variable selection with sparsistency
- Additive regression splines with total variation and non negative garrote penalties
- Smoothing Splines Approximation Using Hilbert Curve Basis Selection
- Improved Estimation of High-dimensional Additive Models Using Subspace Learning
- Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers
- Hierarchical Total Variations and Doubly Penalized ANOVA Modeling for Multivariate Nonparametric Regression
- A group bridge approach for component selection in nonparametric accelerated failure time additive regression model
- Estimating functions and derivatives via adaptive penalized splines
- Bayesian Model Selection in Additive Partial Linear Models Via Locally Adaptive Splines
- Nonparametric Interaction Selection
- Robust variable selection for the varying coefficient model based on composite \(L_1\)-\(L_2\) regression
- Estimation of sparse functional additive models with adaptive group Lasso
- Additive model selection
- Functional Horseshoe Priors for Subspace Shrinkage
- Kernel meets sieve: post-regularization confidence bands for sparse additive model
- Multiple-population shrinkage estimation via sliced inverse regression
- Composite coefficient of determination and its application in ultrahigh dimensional variable screening
- Layer-wise learning strategy for nonparametric tensor product smoothing spline regression and graphical models
This page was built for publication: Component selection and smoothing in multivariate nonparametric regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q869970)
