Estimating a Potential Without the Agony of the Partition Function

DOI10.1137/22M1517135MaRDI QIDQ6148349zbMATH OpenFDO

Authors Eldad Haber, Moshe Eliasof, Luis Tenorio

Publication date 11 January 2024

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2208.09433

zbMATH Keywords

Gibbs densityhyperbolic neural networkMAP estimatepartition functionpotential

Mathematics Subject Classification ID

PDEs with randomness, stochastic partial differential equations (35R60) Probability distributions: general theory (60E05) Numerical optimization and variational techniques (65K10) Finite difference methods for initial value and initial-boundary value problems involving PDEs (65M06) Artificial neural networks and deep learning (68T07)

Abstract: Estimating a Gibbs density function given a sample is an important problem in computational statistics and statistical learning. Although the well established maximum likelihood method is commonly used, it requires the computation of the partition function (i.e., the normalization of the density). This function can be easily calculated for simple low-dimensional problems but its computation is difficult or even intractable for general densities and high-dimensional problems. In this paper we propose an alternative approach based on Maximum A-Posteriori (MAP) estimators, we name Maximum Recovery MAP (MR-MAP), to derive estimators that do not require the computation of the partition function, and reformulate the problem as an optimization problem. We further propose a least-action type potential that allows us to quickly solve the optimization problem as a feed-forward hyperbolic neural network. We demonstrate the effectiveness of our methods on some standard data sets.

Recommendations

• scientific article; zbMATH DE number 6276120
• Stochastic approximation algorithms for partition function estimation of Gibbs random fields
• Algorithms for estimating the partition function of restricted Boltzmann machines
• A Novel Parameter Estimation Method for Boltzmann Machines
• Estimation of non-normalized statistical models by score matching

Cites work

• A Stochastic Approximation Method
• Algorithms for estimating the partition function of restricted Boltzmann machines
• An introduction to data analysis and uncertainty quantification for inverse problems
• Asymptotic methods in statistical decision theory
• Atomic Decomposition by Basis Pursuit
• Collocation Software for Boundary-Value ODEs
• Consistent Estimates Based on Partially Consistent Observations
• Deep convolutional framelets: a general deep learning framework for inverse problems
• Deep neural networks motivated by partial differential equations
• Fully hyperbolic convolutional neural networks
• scientific article; zbMATH DE number 3816970 (Why is no real title available?)
• scientific article; zbMATH DE number 43996 (Why is no real title available?)
• scientific article; zbMATH DE number 847272 (Why is no real title available?)
• scientific article; zbMATH DE number 7646020 (Why is no real title available?)
• Learning regularization functionals a supervised training approach
• Note on the Consistency of the Maximum Likelihood Estimate
• Practical methods for optimal control using nonlinear programming
• Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior
• Stable architectures for deep neural networks
• Theory of statistics

This page was built for publication: Estimating a Potential Without the Agony of the Partition Function

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6148349)