Sparse and efficient estimation for partial spline models with increasing dimension

DOI10.1007/S10463-013-0440-YzbMATH Open1331.65028arXiv1310.8633OpenAlexW2112549647WikidataQ43111864 ScholiaQ43111864MaRDI QIDQ2255168FDOQ2255168

Authors: Guang Cheng, Hao Helen Zhang, Zuofeng Shang 

Publication date: 6 February 2015

Published in: Annals of the Institute of Statistical Mathematics (Search for Journal in Brave)

Abstract: We consider model selection and estimation for partial spline models and propose a new regularization method in the context of smoothing splines. The regularization method has a simple yet elegant form, consisting of roughness penalty on the nonparametric component and shrinkage penalty on the parametric components, which can achieve function smoothing and sparse estimation simultaneously. We establish the convergence rate and oracle properties of the estimator under weak regularity conditions. Remarkably, the estimated parametric components are sparse and efficient, and the nonparametric component can be estimated with the optimal rate. The procedure also has attractive computational properties. Using the representer theory of smoothing splines, we reformulate the objective function as a LASSO-type problem, enabling us to use the LARS algorithm to compute the solution path. We then extend the procedure to situations when the number of predictors increases with the sample size and investigate its asymptotic properties in that context. Finite-sample performance is illustrated by simulations.

Full work available at URL: https://arxiv.org/abs/1310.8633

Recommendations

• scientific article; zbMATH DE number 6951486
• Estimation and variable selection for generalized additive partial linear models
• Spline estimator for ultra-high dimensional partially linear varying coefficient models
• Automatic model selection for partially linear models
• Variable selection for the partial linear single-index model

zbMATH Keywords

semiparametric modelshigh dimensionalityoracle propertyRKHSshrinkage methodssmoothing splinessolution path

Mathematics Subject Classification ID

Ridge regression; shrinkage estimators (Lasso) (62J07) Generalized linear models (logistic models) (62J12)

Cites Work

• Semiparametric Regression
• Smoothing spline ANOVA models
• The Adaptive Lasso and Its Oracle Properties
• Least angle regression. (With discussion)
• Weak convergence and empirical processes. With applications to statistics
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• Simultaneous analysis of Lasso and Dantzig selector
• Title not available (Why is that?)
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• Title not available (Why is that?)
• Some results on Tchebycheffian spline functions and stochastic processes
• Shrinkage tuning parameter selection with a diverging number of parameters
• Adaptive Lasso for sparse high-dimensional regression models
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Tuning parameter selectors for the smoothly clipped absolute deviation method
• New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis
• Penalized quasi-likelihood estimation in partial linear models
• Nonconcave penalized likelihood with a diverging number of parameters.
• Better Subset Regression Using the Nonnegative Garrote
• Automatic model selection for partially linear models
• Asymptotic behavior of M-estimators of p regression parameters when \(p^ 2/n\) is large. I. Consistency
• On the adaptive elastic net with a diverging number of parameters
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Mathematical Statistics
• Adaptive Lasso for Cox's proportional hazards model
• Convergence rates for partially splined models
• An elementary estimator of the partial linear model
• Local and global asymptotic inference in smoothing spline models

Cited In (4)

• Minimax optimal estimation in partially linear additive models under high dimension
• Semiparametric efficient estimation in high-dimensional partial linear regression models
• A partially linear framework for massive heterogeneous data
• An RKHS-based approach to double-penalized regression in high-dimensional partially linear models

Uses Software

• gss
• SemiPar

This page was built for publication: Sparse and efficient estimation for partial spline models with increasing dimension

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2255168)