Nonparametric sparsity and regularization
From MaRDI portal
Abstract: In this work we are interested in the problems of supervised learning and variable selection when the input-output dependence is described by a nonlinear function depending on a few variables. Our goal is to consider a sparse nonparametric model, hence avoiding linear or additive models. The key idea is to measure the importance of each variable in the model by making use of partial derivatives. Based on this intuition we propose a new notion of nonparametric sparsity and a corresponding least squares regularization scheme. Using concepts and results from the theory of reproducing kernel Hilbert spaces and proximal methods, we show that the proposed learning algorithm corresponds to a minimization problem which can be provably solved by an iterative procedure. The consistency properties of the obtained estimator are studied both in terms of prediction and selection performance. An extensive empirical analysis shows that the proposed method performs favorably with respect to the state-of-the-art methods.
Recommendations
Cited in
(59)- Non-linear dictionary learning with partially labeled data
- Variable selection of high-dimensional non-parametric nonlinear systems by derivative averaging to avoid the curse of dimensionality
- Ranking the importance of variables in nonlinear system identification
- Regularizers for structured sparsity
- Convergence of stochastic proximal gradient algorithm
- Statistical modeling of longitudinal data with non-ignorable non-monotone missingness with semiparametric Bayesian and machine learning components
- Learning sparse conditional distribution: an efficient kernel-based approach
- Improved spectral convergence rates for graph Laplacians on \(\varepsilon \)-graphs and \(k\)-NN graphs
- Nonconvex regularization for sparse neural networks
- Kernel variable selection for multicategory support vector machines
- A unified penalized method for sparse additive quantile models: an RKHS approach
- A random block-coordinate Douglas-Rachford splitting method with low computational complexity for binary logistic regression
- The convergence rate of semi-supervised regression with quadratic loss
- Statistical inference in compound functional models
- Variable selection based on squared derivative averages
- Randomized pick-freeze for sparse Sobol indices estimation in high dimension
- Regularity properties for sparse regression
- Sparsity-enforcing regularisation and ISTA revisited
- The performance of semi-supervised Laplacian regularized regression with the least square loss
- Bayesian approximate kernel regression with variable selection
- Norm sensitivity of sparsity regularization with respect to p
- Low-rank and sparse dictionary learning
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
- scientific article; zbMATH DE number 6982912 (Why is no real title available?)
- Statistical sparsity
- Proximal methods for the latent group lasso penalty
- A Fast Gradient Method for Nonnegative Sparse Regression With Self-Dictionary
- Sparse Signal Approximation via Nonseparable Regularization
- Sparse Regularization via Convex Analysis
- Robust and discriminative dictionary learning for face recognition
- Kernel based approaches to local nonlinear non-parametric variable selection
- scientific article; zbMATH DE number 7255176 (Why is no real title available?)
- Sparse and nonnegative sparse D-MORPH regression
- Nonsparse learning with latent variables
- Isotropic non-Lipschitz regularization for sparse representations of random fields on the sphere
- Proximal Gradient Methods for Machine Learning and Imaging
- A maximum principle argument for the uniform convergence of graph Laplacian regressors
- Lipschitz regularity of graph Laplacians on random data clouds
- Efficient kernel-based variable selection with sparsistency
- Improvement on LASSO-type estimator in nonparametric regression
- Thresholding gradient methods in Hilbert spaces: support identification and linear convergence
- Kernel meets sieve: post-regularization confidence bands for sparse additive model
- Variable selection for nonparametric learning with power series kernels
- Performance analysis of the LapRSSLG algorithm in learning theory
- ADMM Algorithmic Regularization Paths for Sparse Statistical Machine Learning
- A General Framework of Nonparametric Feature Selection in High-Dimensional Data
- The Geometry of Sparse Analysis Regularization
- Structure learning via unstructured kernel-based M-estimation
- High-dimensional local linear regression under sparsity and convex losses
- Nonparametric augmented probability weighting with sparsity
- Deep networks for system identification: a survey
- Efficient learning of nonparametric directed acyclic graph with statistical guarantee
- Enhanced feature learning via regularisation: integrating neural networks and kernel methods
- A combined estimator for nonlinear system identification via LPV approximations
- Gradient-induced variable selection in reproducing kernel Hilbert space for survival analysis
- Nonparametric linear feature learning in regression through regularisation
- Sparsity information and regularization in the horseshoe and other shrinkage priors
- A classification-oriented dictionary learning model: explicitly learning the particularity and commonality across categories
- Sparse representation based Fisher discrimination dictionary learning for image classification
This page was built for publication: Nonparametric sparsity and regularization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2933860)
