Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems

DOI10.1214/aos/1176348385zbMath0753.62003OpenAlexW2155969596MaRDI QIDQ1184223

Publication date: 28 June 1992

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1214/aos/1176348385

zbMATH Keywords

least squares image reconstruction linear constraints selection rule minimum discrimination information Bregman's divergences composition consistency f-divergences logically consistent inference method of maximum entropy nonlinear projection nonsymmetric distance prior guess

Mathematics Subject Classification ID

Foundations and philosophical topics in statistics (62A01) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Statistical aspects of information-theoretic topics (62B10)

Related Items

Path-Based Divergence Rates and Lagrangian Uncertainty in Stochastic Flows ⋮ Reporting Bayes factors or probabilities to decision makers of unknown loss functions ⋮ A simple robust asset pricing model under statistical ambiguity ⋮ Bayesian revision of a prior given prior-data conflict, expert opinion, or a similar insight: a large-deviation approach ⋮ Maximum entropy derived and generalized under idempotent probability to address Bayes-frequentist uncertainty and model revision uncertainty: an information-theoretic semantics for possibility theory ⋮ Independence versus indetermination: basis of two canonical clustering criteria ⋮ Stochastic composition optimization of functions without Lipschitz continuous gradient ⋮ Independent Nonlinear Component Analysis ⋮ Reconstructing the image scanning microscopy dataset: an inverse problem ⋮ Traffic rate network tomography with higher‐order cumulants ⋮ Estimating Field-Level Rotations as Dynamic Cycles ⋮ Multidimensional Rational Covariance Extension with Approximate Covariance Matching ⋮ Entropy concentration and the empirical coding game ⋮ Re-examination of Bregman functions and new properties of their divergences ⋮ Estimation of cost allocation coefficients at the farm level using an entropy approach ⋮ Optimization-Based Calibration of Simulation Input Models ⋮ Iterations of paracontractions and firmaly nonexpansive operators with applications to feasibility and optimization ⋮ Inverse problems in biomedical imaging: modeling and methods of solution ⋮ Lagrangian Uncertainty Quantification and Information Inequalities for Stochastic Flows ⋮ DATA SET MODELS AND EXPONENTIAL FAMILIES IN STATISTICAL PHYSICS AND BEYOND ⋮ A simultaneous estimation and variable selection rule ⋮ Maximum entropy and Bayesian approaches to the ratio problem ⋮ Generalized Sinkhorn Iterations for Regularizing Inverse Problems Using Optimal Mass Transport ⋮ Bregman Voronoi diagrams ⋮ Bounded perturbation resilience of projected scaled gradient methods ⋮ A Small-Sample Estimator for the Sample-Selection Model ⋮ Sparsity promoting regularization for effective noise suppression in SPECT image reconstruction ⋮ APPROXIMATING SMALLEST ENCLOSING BALLS WITH APPLICATIONS TO MACHINE LEARNING ⋮ Accelerated alternating minimization algorithm for Poisson noisy image recovery ⋮ Stochastic Control Liaisons: Richard Sinkhorn Meets Gaspard Monge on a Schrödinger Bridge ⋮ Multimarginal Optimal Transport with a Tree-Structured Cost and the Schrödinger Bridge Problem ⋮ Quantum collapse rules from the maximum relative entropy principle ⋮ Moderating probability distributions for unrepresented uncertainty: Application to sentiment analysis via deep learning ⋮ Multi-Domain Regularization Based Computed Tomography for High-Speed Rotation Objects ⋮ Generalized cross entropy method for estimating joint distribution from incomplete information ⋮ Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory ⋮ Inverse problems explicit and implicit formulations with applications in engineering, biophysics and biotechnology ⋮ A foundational approach to generalising the maximum entropy inference process to the multi-agent context ⋮ Estimation of asset demands by heterogeneous agents ⋮ \textit{Maxallent}: maximizers of all entropies and uncertainty of uncertainty ⋮ A multiprojection algorithm using Bregman projections in a product space ⋮ Maximally representative allocations for guaranteed delivery advertising campaigns ⋮ A weighted generalized maximum entropy estimator with a data-driven weight ⋮ Best probability density function for random sampled data ⋮ Updating a progic ⋮ Poisson Noise Reduction with Higher-Order Natural Image Prior Model ⋮ A maximum entropy approach to estimation and inference in dynamic models or Counting fish in the sea using maximum entropy ⋮ Generalized projections for non-negative functions ⋮ Bayesian methods and maximum entropy for ill-posed inverse problems ⋮ A Bayesian approach to diagnosis of asset pricing models ⋮ Speckle noise removal in ultrasound images by first- and second-order total variation ⋮ A fast algorithm for deconvolution and Poisson noise removal ⋮ Non-negatively constrained image deblurring with an inexact interior point method ⋮ Estimation and inference with censored and ordered multinomial response data ⋮ Entropy, divergence and distance measures with econometric applications ⋮ Restoring Poissonian images by a combined first-order and second-order variation approach ⋮ \(M^2\)-spectral estimation: a relative entropy approach ⋮ The geometry of proper scoring rules ⋮ Variational image segmentation model coupled with image restoration achievements ⋮ A feasible direction method for image restoration ⋮ Deformed statistics Kullback-Leibler divergence minimization within a scaled Bregman framework ⋮ Quadratic growth conditions and uniqueness of optimal solution to Lasso ⋮ Testing statistical hypotheses based on the density power divergence ⋮ Traversing the Schrödinger bridge strait: Robert Fortet's marvelous proof redux ⋮ The Pólya information divergence ⋮ Primal and dual Bregman methods with application to optical nanoscopy ⋮ Blending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing ⋮ On the measurability of knowledge acquisition and query processing. ⋮ Probabilism, entropies and strictly proper scoring rules ⋮ Optimal measures and Markov transition kernels ⋮ Comparing dynamic equilibrium models to data: a Bayesian approach ⋮ Inference after checking multiple Bayesian models for data conflict and applications to mitigating the influence of rejected priors ⋮ Inexact version of Bregman proximal gradient algorithm ⋮ Robust Statistical Engineering by Means of Scaled Bregman Distances ⋮ NPtool: a MATLAB software for nonnegative image restoration with Newton projection methods ⋮ A novel sparsity reconstruction method from Poisson data for 3D bioluminescence tomography ⋮ Statistical multiresolution Dantzig estimation in imaging: fundamental concepts and algorithmic framework ⋮ A note on the inevitability of maximum entropy ⋮ Speckle reduction with trained nonlinear diffusion filtering ⋮ Tsallis distribution as a standard maximum entropy solution with `tail' constraint ⋮ Proper local scoring rules ⋮ Iterative regularization algorithms for constrained image deblurring on graphics processors ⋮ Income distributions and decomposable divergence measures ⋮ Quasi-Newton projection methods and the discrepancy principle in image restoration ⋮ On new developments in divergence statistics ⋮ Deblurring Poisson noisy images by total variation with overlapping group sparsity ⋮ Total Variation Restoration of Images Corrupted by Poisson Noise with Iterated Conditional Expectations ⋮ Algorithms for non-negatively constrained maximum penalized likelihood reconstruction in tomographic imaging ⋮ Bregman divergence as relative operator entropy ⋮ Conditional logic and the Principle of Entropy ⋮ Second order total generalized variation for Speckle reduction in ultrasound images ⋮ Continuous analog of accelerated OS-EM algorithm for computed tomography ⋮ Sticky information and model uncertainty in survey data on inflation expectations ⋮ Assessing misspecified asset pricing models with empirical likelihood estimators ⋮ Solution to a function equation and divergence measures ⋮ Cramér-Rao lower bounds arising from generalized Csiszár divergences ⋮ Common Lyapunov function based on Kullback-Leibler divergence for a switched nonlinear system ⋮ Continuity of the maximum-entropy inference: Convex geometry and numerical ranges approach ⋮ Mathematics of Electron Tomography ⋮ Approximation of nonnegative systems by moving averages of fixed order ⋮ Removing multiplicative noise by Douglas-Rachford splitting methods ⋮ Point estimation with exponentially tilted empirical likelihood ⋮ On the linear convergence of forward-backward splitting method. I: Convergence analysis ⋮ Iterative methods for the reconstruction of astronomical images with high dynamic range ⋮ Continuity of the maximum-entropy inference ⋮ Projection theorems and estimating equations for power-law models ⋮ MEASURING DISTRIBUTION MODEL RISK ⋮ Symmetric measures via moments ⋮ High-order total variation-based Poissonian image deconvolution with spatially adapted regularization parameter ⋮ Conditional principles for random weighted measures ⋮ Accelerated Bregman proximal gradient methods for relatively smooth convex optimization ⋮ Fastest rates for stochastic mirror descent methods ⋮ Sketching information divergences ⋮ Bregman divergences in the \((m\times k)\)-partitioning problem ⋮ A maximum entropy approach to the realizability of spin correlation matrices ⋮ Tomographic inverse problem with estimating missing projections ⋮ Continuous-time method and its discretization to inverse problem of intensity-modulated radiation therapy treatment planning ⋮ In defense of the maximum entropy inference process ⋮ Aggregating incoherent agents who disagree ⋮ The General Form of γ-Family of Quantum Relative Entropies ⋮ On the dual of linear inverse problems ⋮ On some entropy functionals derived from Rényi information divergence ⋮ Approximation of linear programs by Bregman's \(D_F\) projections ⋮ Image Denoising via Multiscale Nonlinear Diffusion Models ⋮ Dose-volume constrained optimization in intensity-modulated radiation therapy treatment planning ⋮ Comparison of maximum entropy and higher-order entropy estimators. ⋮ Information theoretic measures of the income distribution in food demand ⋮ Scaling techniques for gradient projection-type methods in astronomical image deblurring ⋮ Nonsymmetric equal sacrifice solutions for claim problem. ⋮ A reweighted \(\ell^2\) method for image restoration with Poisson and mixed Poisson-Gaussian noise ⋮ Empirical entropy for right censored data ⋮ Alternating direction method for the high-order total variation-based Poisson noise removal problem ⋮ Minimization with respect to divergences and applications ⋮ Optimal transport with some directed distances