An equation-by-equation method for solving the multidimensional moment constrained maximum entropy problem

DOI10.2140/CAMCOS.2018.13.189MaRDI QIDQ1789236zbMATH OpenOpenAlexFDO

Authors Wenrui Hao, John Harlim

Publication date 10 October 2018

Published in Communications in Applied Mathematics and Computational Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1702.02242

zbMATH Keywords

equation-by-equation methodhomotopy continuationmaximum entropymoment constrained

Mathematics Subject Classification ID

Newton-type methods (49M15) Numerical computation of solutions to systems of equations (65H10) Global methods, including homotopy approaches to the numerical solution of nonlinear equations (65H20) Measures of information, entropy (94A17)

Abstract: An equation-by-equation (EBE) method is proposed to solve a system of nonlinear equations arising from the moment constrained maximum entropy problem of multidimensional variables. The design of the EBE method combines ideas from homotopy continuation and Newton's iterative methods. Theoretically, we establish the local convergence under appropriate conditions and show that the proposed method, geometrically, finds the solution by searching along the surface corresponding to one component of the nonlinear problem. We will demonstrate the robustness of the method on various numerical examples, including: (1) A six-moment one-dimensional entropy problem with an explicit solution that contains components of order $10^0-10^3$ in magnitude; (2) Four-moment multidimensional entropy problems with explicit solutions where the resulting systems to be solved ranging from $70-310$ equations; (3) Four- to eight-moment of a two-dimensional entropy problem, which solutions correspond to the densities of the two leading EOFs of the wind stress-driven large-scale oceanic model. In this case, we find that the EBE method is more accurate compared to the classical Newton's method, the MATLAB generic solver, and the previously developed BFGS-based method, which was also tested on this problem. (4) Four-moment constrained of up to five-dimensional entropy problems which solutions correspond to multidimensional densities of the components of the solutions of the Kuramoto-Sivashinsky equation. For the higher dimensional cases of this example, the EBE method is superior because it automatically selects a subset of the prescribed moment constraints from which the maximum entropy solution can be estimated within the desired tolerance. This selection feature is particularly important since the moment constrained maximum entropy problems do not necessarily have solutions in general.

Review text

A novel equation solver is introduced that can be used to solve systems of nonlinear equations arising from the moment constrained maximum entropy problem of multidimensional variables. The proposed method, which is called the equation-by-equation (EBE) method, is an iterative method that solves a one-dimensional problem at the first iterate, a two-dimensional problem at the second iterate, a three-dimensional problem at the third iterate, and eventually solves the full system of nonlinear equations corresponding to the maximum entropy problem at the last iterate. Technically, this method combines Newton's method with ideas from homotopy continuation. It is shown that the EBE method is locally convergent under appropriate conditions. Furthermore, sufficient conditions for its global convergence are provided. Through the convergence analysis, it is shown that, geometrically, the proposed method finds the solution of the nonlinear system of equations by tracking along the surface corresponding to one component of the system of nonlinear equations. The EBE method automatically selects a subset of the prescribed constraints from which the maximum entropy solution can be estimated within the desired tolerance. This is an important feature since maximum entropy problems do not necessarily have solutions for general sets of moment constraints. The robustness of the method is demonstrated with various numerical examples. In addition, the new procedure is compared with Newton's method and other numerical methods to show its efficiency.

Recommendations

• The multidimensional maximum entropy moment problem: a review on numerical methods
• Publication:4937618
• The multidimensional moment-constrained maximum entropy problem: A BFGS algorithm with constraint scaling
• scientific article; zbMATH DE number 1159216
• Solution to boundary value problems using the method of maximum entropy

Cites work

• A fast Newton algorithm for entropy maximization in phase determination
• An improved algorithm for the multidimensional moment-constrained maximum entropy problem
• Calculation of maximum entropy densities with application to income distribution
• Continuation along bifurcation branches for a tumor model with a necrotic core
• Fourth-Order Time-Stepping for Stiff PDEs
• Information Theory and Statistical Mechanics
• Is Gauss Quadrature Better than Clenshaw–Curtis?
• Maximum entropy in the finite Stieltjes and Hamburger moment problem
• Newton's method with deflation for isolated singularities of polynomial systems
• Numerical integration using sparse grids
• Numerically solving polynomial systems with Bertini
• On Estimation of a Probability Density Function and Mode
• Quantifying predictability through information theory: small sample estimation in a non-Gaussian framework
• Remarks on Some Nonparametric Estimates of a Density Function
• The multidimensional maximum entropy moment problem: a review on numerical methods
• The multidimensional moment-constrained maximum entropy problem: A BFGS algorithm with constraint scaling

Cited in

(9)

• The multidimensional moment-constrained maximum entropy problem: A BFGS algorithm with constraint scaling
• A stochastic homotopy tracking algorithm for parametric systems of nonlinear equations
• Linear response based parameter estimation in the presence of model error
• A gradient descent method for solving a system of nonlinear equations
• A parameter estimation method using linear response statistics: Numerical scheme
• A homotopy training algorithm for fully connected neural networks
• Greedy randomized sampling nonlinear Kaczmarz methods
• Solving systems of nonlinear equations combining radial basis function neural network and Newton's method
• The multidimensional maximum entropy moment problem: a review on numerical methods

Describes a project that uses

Uses Software

• Bertini

This page was built for publication: An equation-by-equation method for solving the multidimensional moment constrained maximum entropy problem

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1789236)