On a Phase Transition in General Order Spline Regression

DOI10.1109/TIT.2022.3150253MaRDI QIDQ5088529zbMATH OpenOpenAlexFDO

Authors Yandi Shen, Qiyang Han, Fang Han

Publication date 13 July 2022

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2004.10922

zbMATH Keywords

fundamental limitgeneralized splines with free knotsstatistical estimation

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: In the Gaussian sequence model $Y=het{a}_0+varepsilon$ in $mathbb{R}^n$, we study the fundamental limit of approximating the signal $het{a}_0$ by a class $Theta(d,d_0,k)$ of (generalized) splines with free knots. Here $d$ is the degree of the spline, $d_0$ is the order of differentiability at each inner knot, and $k$ is the maximal number of pieces. We show that, given any integer $dgeq0$ and $d_{0}in-1,0,ldots,d-1$, the minimax rate of estimation over $Theta(d,d_0,k)$ exhibits the following phase transition: �egin{equation*} �egin{aligned} inf_{widetilde{ heta}}sup_{ hetainTheta(d,d_0, k)}mathbb{E}_ heta|widetilde{ heta} - heta|^2 asymp_d �egin{cases} kloglog(16n/k), & 2leq kleq k_0,\ klog(en/k), & k geq k_0+1. end{cases} end{aligned} end{equation*} The transition boundary $k_0$, which takes the form $lfloor(d+1)/(d-d_0)floor+1$, demonstrates the critical role of the regularity parameter $d_0$ in the separation between a faster $loglog(16n)$ and a slower $log(en)$ rate. We further show that, once encouraging an additional '$d$-monotonicity' shape constraint (including monotonicity for $d=0$ and convexity for $d=1$), the above phase transition is eliminated and the faster $kloglog(16n/k)$ rate can be achieved for all $k$. These results provide theoretical support for developing $el{l}_0$-penalized (shape-constrained) spline regression procedures as useful alternatives to $el{l}_1$- and $el{l}_2$-penalized ones.

Cited in

(3)

• Transfer learning for piecewise-constant mean estimation: optimality, _1 and _0 penalization
• Fast and optimal inference for change points in piecewise polynomials via differencing
• Online jump and kink detection in segmented linear regression: statistical optimality meets computational efficiency

This page was built for publication: On a Phase Transition in General Order Spline Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5088529)