Degrees of freedom in submodular regularization: a computational perspective of Stein's unbiased risk estimate

DOI10.1016/J.JMVA.2019.104546MaRDI QIDQ2293382zbMATH OpenOpenAlexWikidataFDO

Authors Kentaro Minami

Publication date 5 February 2020

Published in Journal of Multivariate Analysis (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/j.jmva.2019.104546

zbMATH Keywords

degrees of freedomfused LassoStein's unbiased risk estimatestructured sparse estimationsubmodular function

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Review text

This paper studies regularized estimators obtained by means of two polyhedral convex classes of regularizators for the regression model defined through submodular functions, the Lovász extension regularization and submodular norm regularization. Characterizations are obtained for the degrees of freedom of the submodular regularization estimators. It is shown that their degrees of freedom can be represented in terms of partitions induced by the estimators. Unified representations of the degrees of freedom are derived, which are independent of the choices of the design matrix and of the penalty function. If the design matrix has full column rank, calculating an unbiased estimator of the degrees of freedom requires an additional computational cost of only \(O(p \log p)\) after a solution for the estimator is obtained, for \(p\) the dimension of the parameter. Relationships between submodular regularization estimators and projection and anti-projection estimators are established. The submodular regularization estimators are shown to be special cases of anti-projection estimators with respect to certain classes of polyhedra, so that general results for anti-projection estimators are valid for them. Specific examples of submodular regularization estimators are studied and numerical simulations are conducted to investigate their empirical performance.

Recommendations

• The degrees of freedom of partly smooth regularizers
• Degrees of freedom for piecewise Lipschitz estimators
• On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression
• Computing the degrees of freedom of rank-regularized estimators and cousins
• On the degrees of freedom of reduced-rank estimators in multivariate regression

Cites work

• Consistency of cross validation for comparing regression procedures
• Degrees of freedom in lasso problems
• Estimating the dimension of a model
• Estimation and testing under sparsity. École d'Été de Probabilités de Saint-Flour XLV -- 2015
• Estimation of the mean of a multivariate normal distribution
• Excess optimism: how biased is the apparent error of an estimator tuned by SURE?
• scientific article; zbMATH DE number 193111 (Why is no real title available?)
• scientific article; zbMATH DE number 3444596 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 3365044 (Why is no real title available?)
• Learning with submodular functions: a convex optimization perspective
• Model Selection and Estimation in Regression with Grouped Variables
• On the ``degrees of freedom of the lasso
• On the degrees of freedom in shape-restricted regression.
• On the degrees of freedom in shrinkage estimation
• Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
• SLOPE is adaptive to unknown sparsity and asymptotically minimax
• Slope meets Lasso: improved oracle bounds and optimality
• SLOPE-adaptive variable selection via convex optimization
• Some Comments on C P
• Sparsity and Smoothness Via the Fused Lasso
• Structured variable selection with sparsity-inducing norms
• Submodular functions and optimization.
• The Estimation of Prediction Error
• The solution path of the generalized lasso
• Variational Analysis

Cited in

(6)

• Computing the degrees of freedom of rank-regularized estimators and cousins
• The degrees of freedom of partly smooth regularizers
• On Degrees of Freedom of Projection Estimators With Applications to Multivariate Nonparametric Regression
• Stein Unbiased GrAdient estimator of the Risk (SUGAR) for Multiple Parameter Selection
• Pattern recovery and signal denoising by SLOPE when the design matrix is orthogonal
• A unified framework for pattern recovery in penalized and thresholded estimation and its geometry

This page was built for publication: Degrees of freedom in submodular regularization: a computational perspective of Stein's unbiased risk estimate

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2293382)