Enhanced feature learning via regularisation: integrating neural networks and kernel methods
This interesting paper studies enhanced feature learning via regularization integrating neural networks and kernel methods. To set the scene, in the case of high dimensional data analysis, feature selection methods are important. In this regard, in order to extract meaningful information from complex and high dimensional data sets, representation learning often aims to automate this process. In high dimensions, however, non-parametric methods often struggle which makes multi-index models, a good alternative method. The authors propose a new method for feature learning and function estimation in supervised learning via regularised empirical risk minimisation considering functions as certain expectations of some smooth function classes over all possible one dimensional projections of the data. The authors, in particular, construct a gradient-based computational method for the above suitably scaled expectation using kernel neural networks for particles to approximate the above expectation, where the positive homogeneity of the Brownian kernel leads to improved robustness to local minima. Numerical experiments support the authors complexity estimates given in the paper.\N\NThe paper is well written with a good set of references.
- 10.1162/153244303321897690
- 10.1162/1532443041424300
- A Multiple-Index Model and Dimension Reduction
- A new algorithm for estimating the effective dimension-reduction subspace
- An Adaptive Estimation of Dimension Reduction Space
- An Introduction to Numerical Analysis
- Banach space representer theorems for neural networks and ridge splines
- Bounds on rates of variable-basis and neural-network approximation
- Breaking the curse of dimensionality with convex neural networks
- Concentration inequalities. A nonasymptotic theory of independence
- Convex multi-task feature learning
- Distance-based classification with Lipschitz functions
- Embeddings between Barron spaces with higher-order activation functions
- First-order methods in optimization
- Gaussian processes for machine learning.
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- High-dimensional probability. An introduction with applications in data science
- High-dimensional statistics. A non-asymptotic viewpoint
- scientific article; zbMATH DE number 1804115 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- Kernel dimension reduction in regression
- Kernel ridge regression
- Learning the kernel matrix with semidefinite programming
- Mean field analysis of neural networks: a law of large numbers
- Model Selection and Estimation in Regression with Grouped Variables
- Nearly unbiased variable selection under minimax concave penalty
- Nonparametric sparsity and regularization
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- On the Nyström method for approximating a gram matrix for improved kernel-based learning
- Optimization with sparsity-inducing penalties
- Rademacher Chaos Complexities for Learning the Kernel Problem
- Scikit-learn: machine learning in Python
- Selected Works of David Brillinger
- Sliced Inverse Regression for Dimension Reduction
- Theory and statistical applications of stochastic processes
- Theory of Reproducing Kernels
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
This page was built for publication: Enhanced feature learning via regularisation: integrating neural networks and kernel methods
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6887367)
