Greedy algorithms for prediction
From MaRDI portal
Publication:265302
DOI10.3150/14-BEJ691zbMATH Open1388.62209arXiv1602.01951OpenAlexW3103732143MaRDI QIDQ265302FDOQ265302
Authors: Alessio Sancetta 
Publication date: 1 April 2016
Published in: Bernoulli (Search for Journal in Brave)
Abstract: In many prediction problems, it is not uncommon that the number of variables used to construct a forecast is of the same order of magnitude as the sample size, if not larger. We then face the problem of constructing a prediction in the presence of potentially large estimation error. Control of the estimation error is either achieved by selecting variables or combining all the variables in some special way. This paper considers greedy algorithms to solve this problem. It is shown that the resulting estimators are consistent under weak conditions. In particular, the derived rates of convergence are either minimax or improve on the ones given in the literature allowing for dependence and unbounded regressors. Some versions of the algorithms provide fast solution to problems such as Lasso.
Full work available at URL: https://arxiv.org/abs/1602.01951
Recommendations
- 10.1162/153244304773936108
- Approximation and learning by greedy algorithms
- On the selection of predictors by using greedy algorithms and information theoretic criteria
- Coordinate descent algorithms for lasso penalized regression
- On the consistency of feature selection using greedy least squares regression
Density estimation (62G07) Asymptotic properties of nonparametric inference (62G20) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)
Cites Work
- Nearly unbiased variable selection under minimax concave penalty
- Least angle regression. (With discussion)
- Pathwise coordinate optimization
- Title not available (Why is that?)
- Statistics for high-dimensional data. Methods, theory and applications.
- Convergence of a block coordinate descent method for nondifferentiable minimization
- A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training
- High-dimensional generalized linear models and the lasso
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Universal approximation bounds for superpositions of a sigmoidal function
- Learning Theory and Kernel Machines
- High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity
- On asymptotically optimal confidence regions and tests for high-dimensional models
- A new mixing notion and functional central limit theorems for a sieve bootstrap in time series
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Sparsity oracle inequalities for the Lasso
- \(\ell_1\)-penalized quantile regression in high-dimensional sparse models
- Confidence sets in sparse regression
- Boosting for high-dimensional linear models
- Sparse models and methods for optimal instruments with an application to eminent domain
- Non-strong mixing autoregressive processes
- Statistical significance in high-dimensional linear models
- Degrees of freedom in lasso problems
- Matching pursuits with time-frequency dictionaries
- Introduction to nonparametric estimation
- Gaussian model selection
- On Measuring and Correcting the Effects of Data Mining and Model Selection
- Boosting With theL2Loss
- Basic properties of strong mixing conditions. A survey and some open questions
- Mixing properties of ARMA processes
- Aggregation for Gaussian regression
- On the ``degrees of freedom of the lasso
- Approximation and learning by greedy algorithms
- DATA-DEPENDENT ESTIMATION OF PREDICTION FUNCTIONS
- A cross-validatory method for dependent data
- Sieve Extremum Estimates for Weakly Dependent Data
- Forecast combination across estimation windows
- Smoothing Parameter Selection in Nonparametric Regression Using an Improved Akaike Information Criterion
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
- A dynamic model of expected bond returns: A functional gradient descent approach
- Invariance principles for absolutely regular empirical processes
- Two lower estimates in greedy approximation
- A new weak dependence condition and applications to moment inequalities
- Regular variation of GARCH processes.
- Minimax estimation via wavelet shrinkage
- Maximal inequalities via bracketing with adaptive truncation
- Some remarks on greedy algorithms
- \(\ell _{1}\)-regularized linear regression: persistence and oracle inequalities
- Asymptotic theory of weakly dependent stochastic processes
- Weak greedy algorithms
- On the uniform convergence of empirical norms and inner products, with application to causal inference
- A new covariance inequality and applications.
- Rate of convergence of pure greedy algorithm.
- On the consistency of feature selection using greedy least squares regression
- Splines for financial volatility
- Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm
- Greedy approximation
- Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation
- Risk bounds for mixture density estimation
- A maximal 𝕃_{𝕡}-inequality for stationary sequences and its applications
- Forecasting Time Series Subject to Multiple Structural Breaks
- A NONPARAMETRIC ESTIMATOR FOR THE COVARIANCE FUNCTION OF FUNCTIONAL DATA
- Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise
- A Recursive Algorithm for Mixture of Densities Estimation
- Title not available (Why is that?)
- Best subset selection, persistence in high-dimensional statistical learning and optimization under \(l_1\) constraint
- Bootstrap model selection for possibly dependent and heterogeneous data
- Propriétés de mélange des processus autorégressifs polynomiaux. (Mixing properties of polynomial autoregressive processes)
Cited In (7)
- Efficient prediction algorithms for binary decomposition techniques
- 10.1162/153244304773936108
- ESTIMATION FOR THE PREDICTION OF POINT PROCESSES WITH MANY COVARIATES
- On the selection of predictors by using greedy algorithms and information theoretic criteria
- Semiparametric estimation of plane similarities: application to fast computation of aeronautic loads
- Algorithms with Predictions
- On the consistency of feature selection using greedy least squares regression
This page was built for publication: Greedy algorithms for prediction
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q265302)
