Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
From MaRDI portal
Publication:2039786
Abstract: We consider the problem of nonparametric regression when the covariate is -dimensional, where . In this paper we introduce and study two nonparametric least squares estimators (LSEs) in this setting---the entirely monotonic LSE and the constrained Hardy-Krause variation LSE. We show that these two LSEs are natural generalizations of univariate isotonic regression and univariate total variation denoising, respectively, to multiple dimensions. We discuss the characterization and computation of these two LSEs obtained from data points. We provide a detailed study of their risk properties under the squared error loss and fixed uniform lattice design. We show that the finite sample risk of these LSEs is always bounded from above by modulo logarithmic factors depending on ; thus these nonparametric LSEs avoid the curse of dimensionality to some extent. We also prove nearly matching minimax lower bounds. Further, we illustrate that these LSEs are particularly useful in fitting rectangular piecewise constant functions. Specifically, we show that the risk of the entirely monotonic LSE is almost parametric (at most up to logarithmic factors) when the true function is well-approximable by a rectangular piecewise constant entirely monotone function with not too many constant pieces. A similar result is also shown to hold for the constrained Hardy-Krause variation LSE for a simple subclass of rectangular piecewise constant functions. We believe that the proposed LSEs yield a novel approach to estimating multivariate functions using convex optimization that avoid the curse of dimensionality to some extent.
The authors study isotonic regression with \(d\)-dimensional covariate. The considered model is \(y=f(x) + \eta\), where \(f\) is an unknown regression function and \(\eta\) is a Gaussian random variable, and the goal is to find a nonparametric estimator of \(f\). In particular, the authors propose nonparametric least square estimators of \(f\) if \(f\) is an entirely monotonic function or \(f\) is of bounded Hardy-Krause variation. Since both classes of entirely monotonic functions and functions of bounded HK variation are infinitely dimensional, the authors use non-negative least squares (NNLS), and Lasso approaches to find the estimate of \(f\). Further, the authors provide a detailed study of the risk properties of the estimators and discuss the problem of a possible misspecification of the function \(f\).
Recommendations
- Isotonic regression in general dimensions
- Isotonic regression in multi-dimensional spaces and graphs
- Strictly monotone and smooth nonparametric regression for two or more variables
- Nonparametric least squares estimation of a multivariate convex regression function
- On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression
Cites work
- $\ell_1$ Trend Filtering
- Adaptive piecewise polynomial estimation via trend filtering
- Adaptive rates for total variation image denoising
- Adaptive risk bounds in univariate total variation denoising and trend filtering
- An Empirical Distribution Function for Sampling with Incomplete Information
- An introduction to total variation for image analysis
- Approximate \(\ell_0\)-penalized estimation of piecewise-constant signals on graphs
- Bracketing entropy of high dimensional distributions
- Discrete total variation: new definition and minimization
- Functions of bounded variation, signed measures, and a general Koksma–Hlawka inequality
- Generalization bounds for function approximation from scattered noisy data
- How many Laplace transforms of probability measures are there?
- scientific article; zbMATH DE number 3133643 (Why is no real title available?)
- scientific article; zbMATH DE number 193111 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- scientific article; zbMATH DE number 3390139 (Why is no real title available?)
- scientific article; zbMATH DE number 3108056 (Why is no real title available?)
- Inefficient estimators of the bivariate survival function for three models
- Information-theoretic determination of minimax rates of convergence
- Isotonic regression in general dimensions
- Locally adaptive regression splines
- Maximum Likelihood Estimates of Monotone Parameters
- Metric entropy of high dimensional distributions
- Minimax estimation via wavelet shrinkage
- Multidimensional variation for quasi-Monte Carlo
- Multivariate approximation
- Nonlinear total variation based noise removal algorithms
- Nonparametric estimation under shape constraints. Estimators, algorithms and asymptotics
- Nonparametric shape-restricted regression
- On risk bounds in isotonic and other shape restricted regression problems
- On the degrees of freedom in shape-restricted regression.
- On the prediction performance of the Lasso
- On the total variation for functions of several variables and a multidimensional analog of Helly's selection principle
- On the total variation regularized estimator over a class of tree graphs
- On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
- Optimal global rates of convergence for nonparametric regression
- Polynomial approximation via compressed sensing of high-dimensional functions on lower sets
- Quasi-Monte Carlo filtering in nonlinear dynamic systems
- Risk bounds in isotonic regression
- Sequential detection of three-dimensional signals under dependent noise
- Sharp oracle inequalities for least squares estimators in shape restricted regression
- Smoothing spline ANOVA for exponential families, with application to the Wisconsin epidemiological study of diabetic retinopathy. (The 1994 Neyman Memorial Lecture)
- Sparse grids
- Tensor product space ANOVA models.
- Universal approximation bounds for superpositions of a sigmoidal function
Cited in
(16)- Bayesian non-parametric ordinal regression under a monotonicity constraint
- Confidence intervals for multiple isotonic regression and other monotone models
- Total variation regularized Fréchet regression for metric-space valued data
- Tensor denoising with trend filtering
- Bracketing numbers of convex and m-monotone functions on polytopes
- A study on the least squares estimator of multivariate isotonic regression function
- A cross-validation framework for signal denoising with applications to trend filtering, dyadic CART and beyond
- Noisy linear inverse problems under convex constraints: exact risk asymptotics in high dimensions
- MARS via lasso
- Multivariate trend filtering for lattice data
- GLS under monotone heteroskedasticity
- Convergence rates for estimating multivariate scale mixtures of uniform densities
- A joint estimation approach for monotonic regression functions in general dimensions
- Spatially adaptive online prediction of piecewise regular functions
- Bernstein-based nonparametric estimation of the cross ratio function under univariate right censoring
- Frame-constrained total variation regularization for white noise regression
This page was built for publication: Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2039786)
