Isotonic regression in multi-dimensional spaces and graphs

DOI10.1214/20-AOS1947MaRDI QIDQ1996789zbMATH OpenOpenAlexFDO

Authors Yanyan Li

Publication date 26 February 2021

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1812.08944, https://projecteuclid.org/euclid.aos/1607677251

zbMATH Keywords

adaptive estimationblock estimatorisotonic regressionisotonic regression on graphslattice designmax-min estimatormin-max estimatorminimax ratemultiple isotonic regressionoracle propertyrandom designvariable selection

Mathematics Subject Classification ID

Minimax procedures in statistical decision theory (62C20) Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Probabilistic graphical models (62H22) Response surface designs (62K20)

Abstract: In this paper we study minimax and adaptation rates in general isotonic regression. For uniform deterministic and random designs in $[0,1{]}^d$ with $dge2$ and $N(0,1)$ noise, the minimax rate for the $el{l}_2$ risk is known to be bounded from below by $n^{-1/d}$ when the unknown mean function $f$ is nondecreasing and its range is bounded by a constant, while the least squares estimator (LSE) is known to nearly achieve the minimax rate up to a factor $(logn{)}^{g}amma$ where $n$ is sample size, $gamma=4$ in the lattice design and $gamma=max9/2,(d^2+d+1)/2$ in the random design. Moreover, the LSE is known to achieve the adaptation rate $(K/n{)}^{-2/d}{1veelog(n/K)}^{2gamma}$ when $f$ is piecewise constant on $K$ hyperrectangles in a partition of $[0,1{]}^d$. Due to the minimax theorem, the LSE is identical on every design point to both the max-min and min-max estimators over all upper and lower sets containing the design point. This motivates our consideration of estimators which lie in-between the max-min and min-max estimators over possibly smaller classes of upper and lower sets, including a subclass of block estimators. Under a $q$-th moment condition on the noise, we develop $el{l}_q$ risk bounds for such general estimators for isotonic regression on graphs. For uniform deterministic and random designs in $[0,1{]}^d$ with $dge3$, our $el{l}_2$ risk bound for the block estimator matches the minimax rate $n^{-1/d}$ when the range of $f$ is bounded and achieves the near parametric adaptation rate $(K/n){1veelog(n/K)}^d$ when $f$ is $K$-piecewise constant. Furthermore, the block estimator possesses the following oracle property in variable selection: When $f$ depends on only a subset $S$ of variables, the $el{l}_2$ risk of the block estimator automatically achieves up to a poly-logarithmic factor the minimax rate based on the oracular knowledge of $S$.

Review text

This work considers the minimax and adaptation rates in the multivariate isotonic regression setting. The minimax theorem is used as the theoretical foundation to build a case for developing estimators which lie in-between the max-min and min-max estimators over possibly smaller classes of upper and lower sets, including a subclass of block estimators. Further, based on appropriate moment conditions on the noise, the research proceeds by providing corresponding risk bounds for such general estimators for isotonic regression on graphs. The asymptotic properties of these estimates are studied in detail and in particular the block estimate is shown to possess the so-called oracle property in variable selection.

Recommendations

• Isotonic regression in general dimensions
• Isotonic regression with unknown permutations: statistics, computation and adaptation
• Risk bounds in isotonic regression
• Limit distribution theory for block estimators in multiple isotonic regression
• Isotonic regression for multiple independent variables

Cites work

• An Algorithm for Restricted Least Squares Regression
• An Empirical Distribution Function for Sampling with Incomplete Information
• Contraction and uniform convergence of isotonic regression
• Estimating a regression function
• Hellinger-consistency of certain nonparametric maximum likelihood estimators
• scientific article; zbMATH DE number 3126936 (Why is no real title available?)
• scientific article; zbMATH DE number 193111 (Why is no real title available?)
• scientific article; zbMATH DE number 777878 (Why is no real title available?)
• scientific article; zbMATH DE number 6791300 (Why is no real title available?)
• scientific article; zbMATH DE number 3288992 (Why is no real title available?)
• Isotonic regression for multiple independent variables
• Isotonic regression in general dimensions
• Maximum Likelihood Estimates of Monotone Parameters
• Monotone nonparametric regression with random design
• On Integrated L¹ Convergence Rate of an Isotonic Regression Estimator for Multivariate Observations
• On matrix estimation under monotonicity constraints
• On risk bounds in isotonic and other shape restricted regression problems
• On the \(\mathbb L_p\)-error of monotonicity constrained estimators
• On the degrees of freedom in shape-restricted regression.
• Rates of convergence for minimum contrast estimators
• Risk bounds in isotonic regression
• Sharp oracle inequalities for least squares estimators in shape restricted regression
• The L₂risk of an isotonic estimate

Cited in

(24)

• On the computation and some applications of multivariate isotonic regression
• Risk bounds in isotonic regression
• An isotonic regression problem for infinite-dimensional parameters
• Limit distribution theory for block estimators in multiple isotonic regression
• Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation
• Confidence intervals for multiple isotonic regression and other monotone models
• Adaptive estimation of multivariate piecewise polynomials and bounded variation functions by optimal decision trees
• Isotonic regression with unknown permutations: statistics, computation and adaptation
• On estimation of isotonic piecewise constant signals
• Isotonic regression in general dimensions
• Posterior contraction and testing for multivariate isotonic regression
• Algorithm AS 257: Isotonic Regression for Umbrella Orderings
• Uncoupled isotonic regression via minimum Wasserstein deconvolution
• Coverage of credible intervals in Bayesian multivariate isotonic regression
• Estimation and bootstrap for stochastically monotone Markov processes
• Fused Lasso nearly-isotonic signal approximation in general dimensions
• Monotone Cubic B-Splines with a Neural-Network Generator
• Isotonic subgroup selection
• Characterizing the minimax rate of nonparametric regression under bounded star-shaped constraints
• Spatially adaptive online prediction of piecewise regular functions
• Bayesian inference for multivariate monotone densities
• Distributionally robust risk evaluation with an isotonic constraint
• Optimal confidence bands for shape-restricted regression in multidimensions
• Frame-constrained total variation regularization for white noise regression

This page was built for publication: Isotonic regression in multi-dimensional spaces and graphs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1996789)