Solution of linear ill-posed problems using overcomplete dictionaries

DOI10.1214/16-AOS1445zbMATH Open1346.62061OpenAlexW2963208245MaRDI QIDQ309741FDOQ309741

Authors Marianna Pensky 

Publication date 7 September 2016

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1408.3386

zbMATH Keywords

Lassooracle inequalityadaptive estimationlinear inverse problems

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Bayesian problems; characterization of Bayes procedures (62C10)

Abstract: In the present paper we consider application of overcomplete dictionaries to solution of general ill-posed linear inverse problems. Construction of an adaptive optimal solution for such problems usually relies either on a singular value decomposition or representation of the solution via an orthonormal basis. The shortcoming of both approaches lies in the fact that, in many situations, neither the eigenbasis of the linear operator nor a standard orthonormal basis constitutes an appropriate collection of functions for sparse representation of the unknown function. In the context of regression problems, there have been an enormous amount of effort to recover an unknown function using an overcomplete dictionary. One of the most popular methods, Lasso, is based on minimizing the empirical likelihood and requires stringent assumptions on the dictionary, the, so called, compatibility conditions. While these conditions may be satisfied for the original dictionary functions, they usually do not hold for their images due to contraction imposed by the linear operator. In what follows, we bypass this difficulty by a novel approach which is based on inverting each of the dictionary functions and matching the resulting expansion to the true function, thus, avoiding unrealistic assumptions on the dictionary and using Lasso in a predictive setting. We examine both the white noise and the observational model formulations and also discuss how exact inverse images of the dictionary functions can be replaced by their approximate counterparts. Furthermore, we show how the suggested methodology can be extended to the problem of estimation of a mixing density in a continuous mixture. For all the situations listed above, we provide the oracle inequalities for the risk in a finite sample setting. Simulation studies confirm good computational properties of the Lasso-based technique.

Recommendations

• Solution of linear ill-posed problems using random dictionaries
• Solution of linear ill-posed problems by model selection and aggregation
• An iterative thresholding algorithm for linear inverse problems with a sparsity constraint
• Regularization of some linear ill-posed problems with discretized random noisy data
• scientific article; zbMATH DE number 2163517

Cites work

• scientific article; zbMATH DE number 3839134 (Why is no real title available?)
• scientific article; zbMATH DE number 1391247 (Why is no real title available?)
• Adaptive Laguerre density estimation for mixed Poisson models
• Adaptive Wavelet Galerkin Methods for Linear Inverse Problems
• Adaptive demixing in Poisson mixture models
• Deconvolution problems in nonparametric statistics
• Model Selection and Estimation in Regression with Grouped Variables
• Nonlinear estimation for linear inverse problems with error in the operator
• Nonlinear solution of linear inverse problems by wavelet-vaguelette decomposition
• Nonparametric Estimation of a Mixing Density via the Kernel Method
• On inverse problems with unknown operators
• On the prediction performance of the Lasso
• On universal oracle inequalities related to high-dimensional linear models
• Oracle inequalities and optimal inference under group sparsity
• Oracle inequalities for inverse problems
• Ridgelets: estimating with ridge functions
• Risk hull method and regularization by projections of ill-posed inverse problems
• SPADES and mixture models
• Sharp oracle inequalities for aggregation of affine estimators
• Simultaneous analysis of Lasso and Dantzig selector
• Sparse model selection under heterogeneous noise: exact penalisation and data-driven thresholding
• Sparsity oracle inequalities for the Lasso
• Statistics for high-dimensional data. Methods, theory and applications.
• Thresholding estimators for linear inverse problems and deconvolutions
• Wavelet decomposition approaches to statistical inverse problems

Cited in

(4)

• Solution of linear ill-posed problems by model selection and aggregation
• On the prediction loss of the Lasso in the partially labeled setting
• Solution of linear ill-posed problems using random dictionaries
• Estimation of a delta-contaminated density of a random intensity of Poisson data

This page was built for publication: Solution of linear ill-posed problems using overcomplete dictionaries

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q309741)