Graphical Models, Exponential Families, and Variational Inference

DOI10.1561/2200000001zbMath1193.62107OpenAlexW4293052541MaRDI QIDQ3580906

Martin J. Wainwright, Michael I. Jordan

Publication date: 14 August 2010

Published in: Foundations and Trends® in Machine Learning (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1561/2200000001

Mathematics Subject Classification ID

Multivariate analysis (62H99) Applications of graph theory (05C90) Foundations and philosophical topics in statistics (62A01)

Related Items

A family of CCCP algorithms which minimize the TRW free energy, Entity resolution with empirically motivated priors, Variational inference for count response semiparametric regression, A deep learning algorithm for high-dimensional exploratory item factor analysis, Consistency and generalization bounds for maximum entropy density estimation, A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling, Promote sign consistency in the joint estimation of precision matrices, Variational inference for high dimensional structured factor copulas, On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example, Learning undirected graphical models using persistent sequential Monte Carlo, Variational Hamiltonian Monte Carlo via score matching, A general method for robust Bayesian modeling, Stagewise learning for noisy \(k\)-ary preferences, Parallel finite element technique using Gaussian belief propagation, A unified statistical framework for single cell and bulk RNA sequencing data, Cycle-based cluster variational method for direct and inverse inference, Margin losses for training conditional random fields, Multicuts and perturb \& MAP for probabilistic graph clustering, A probabilistic graphical model approach to stochastic multiscale partial differential equations, A nonparametric belief propagation method for uncertainty quantification with applications to flow in random porous media, A probabilistic graphical model based stochastic input model construction, Fast pseudolikelihood maximization for direct-coupling analysis of protein structure from many homologous amino-acid sequences, Consistency under sampling of exponential random graph models, Optimal smoothing for spherical Gauss-Markov random fields with application to weather data estimation, Variational algorithms for biclustering models, Kullback-Leibler upper confidence bounds for optimal sequential allocation, Multivariate Bernoulli distribution, Accelerated training of max-margin Markov networks with kernels, Predictive coarse-graining, Scalable information inequalities for uncertainty quantification, Spatio-temporal random fields: compressible representation and distributed estimation, Local conditional and marginal approach to parameter estimation in discrete graphical models, Mean field variational Bayes for continuous sparse signal shrinkage: pitfalls and remedies, Online fault diagnosis for nonlinear power systems, Mixed-membership naive Bayes models, Maximum entropy models and subjective interestingness: an application to tiles in binary databases, Model-based multiple rigid object detection and registration in unstructured range data, Harmony potentials fusing global and local scale for semantic image segmentation, Finding hidden cliques of size \(\sqrt{N/e}\) in nearly linear time, Efficient approximate linear programming for factored MDPs, Optimal measures and Markov transition kernels, Stochastic global optimization as a filtering problem, Graphical models for optimal power flow, Beyond inverse Ising model: structure of the analytical solution, An extended Perona-Malik model based on probabilistic models, Image labeling by assignment, Recursive sum-product algorithm for generalized outer-planar graphs, Estimating networks with jumps, Classification and estimation in the stochastic blockmodel based on the empirical degrees, Mean field inference for the Dirichlet process mixture model, Random graphs with a given degree sequence, Penalized wavelets: embedding wavelets into semiparametric regression, Robust graphical modeling of gene networks using classical and alternative \(t\)-distributions, Hybrid tractability of valued constraint problems, Learning multi-linear representations of distributions for efficient inference, A geometric view of conjugate priors, Efficient distributed sensing using adaptive censoring-based inference, \(p\)-value model selection criteria for exponential families of increasing dimension, Robustifying sum-product networks, Efficient belief propagation in second-order Bayesian networks for singly-connected graphs, Message-passing algorithms for inference and optimization, Kernel methods in machine learning, Variational inference for probabilistic Poisson PCA, The variational Garrote, Neural network implementation of inference on binary Markov random fields with probability coding, Asymptotic analysis of estimators on multi-label data, Probabilistic inference of binary Markov random fields in spiking neural networks through mean-field approximation, Relational linear programming, Learning Bayesian network parameters under equivalence constraints, Diagonal stationary points of the Bethe functional, Copula directed acyclic graphs, A survey on learning approaches for undirected graphical models. Application to scene object recognition, Density Estimation via Bayesian Inference Engines, A spatial Markovian framework for estimating regional and local dynamics of annual plants with dormancy, Variational Bayes approach for model aggregation in unsupervised classification with Markovian dependency, High-dimensional structure estimation in Ising models: local separation criterion, The truncated K-moment problem for closure of open sets, The expressive power of binary submodular functions, Loglinear model selection and human mobility, High-dimensional Ising model selection using \(\ell _{1}\)-regularized logistic regression, Hierarchical Dirichlet scaling process, Estimating time-varying networks, Hierarchical relational models for document networks, Logistic Normal Multinomial Factor Analyzers for Clustering Microbiome Data, Updating of the Gaussian graphical model through targeted penalized estimation, Convergence rates for optimised adaptive importance samplers, A new approach for conditioning process-based geologic models to well data, Approximate models and robust decisions, Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation, Variational Bayesian strategies for high-dimensional, stochastic design problems, On generalization in moment-based domain adaptation, A Projection Based Conditional Dependence Measure with Applications to High-dimensional Undirected Graphical Models, Efficient semidefinite branch-and-cut for MAP-MRF inference, A mixture of experts latent position cluster model for social network data, The Bouncy Particle Sampler: A Non-Reversible Rejection-Free Markov Chain Monte Carlo Method, Tuning-Free Heterogeneity Pursuit in Massive Networks, Bootstrapping on undirected binary networks via statistical mechanics, Estimating heterogeneous graphical models for discrete data with an application to roll call voting, The explicit form of expectation propagation for a simple statistical model, Regularized brain reading with shrinkage and smoothing, Restricted Indian buffet processes, Mirrored conditional random field model for object recognition in indoor environments, Factor graph fragmentization of expectation propagation, Graphical models for statistical inference and data assimilation, Deterministic replica-exchange method without pseudo random numbers for simulations of complex systems, Variational Bayesian approximation of inverse problems using sparse precision matrices, Path-space variational inference for non-equilibrium coarse-grained systems, Stochastic representation of FGM copulas using multivariate Bernoulli random variables, Statistical physics and representations in real and artificial neural networks, Bayesian inference of clustering and multiple Gaussian graphical models selection, Computationally efficient Bayesian unit-level models for non-Gaussian data under informative sampling with application to estimation of health insurance coverage, Streamlined mean field variational Bayes for longitudinal and multilevel data analysis, Variational inference with NoFAS: normalizing flow with adaptive surrogate for computationally expensive models, Probabilistic programming with stochastic variational message passing, Structured hierarchical models for probabilistic inference from perturbation screening data, The computational asymptotics of Gaussian variational inference and the Laplace approximation, SHOPPER: a probabilistic model of consumer choice with substitutes and complements, Sparse variational Bayesian approximations for nonlinear inverse problems: applications in nonlinear elastography, Hierarchical estimation of parameters in Bayesian networks, Sparse exponential family principal component analysis, A stochastic variational framework for recurrent Gaussian processes models, Joint estimation of heterogeneous exponential Markov random fields through an approximate likelihood inference, Nonasymptotic estimates for stochastic gradient Langevin dynamics under local conditions in nonconvex optimization, Iterative multiplicative filters for data labeling, Torus graphs for multivariate phase coupling analysis, A global-local approach for detecting hotspots in multiple-response regression, Uniform joint screening for ultra-high dimensional graphical models, Learning a tree-structured Ising model in order to make predictions, \(\alpha\)-variational inference with statistical guarantees, A variational Bayesian approach to identifying whole-brain directed networks with fMRI data, Mean field dynamics of stochastic cellular automata for random and small-world graphs, Model-based clustering of large networks, Structure estimation for discrete graphical models: generalized covariance matrices and their inverses, Theoretical and computational guarantees of mean field variational inference for community detection, Variational Bayesian inference with Gaussian-mixture approximations, Causality and Bayesian network PDEs for multiscale representations of porous media, Data-driven, variational model reduction of high-dimensional reaction networks, Mixed membership stochastic blockmodels for heterogeneous networks, TAP free energy, spin glasses and variational inference, Polyhedral conditions for the nonexistence of the MLE for hierarchical log-linear models, Probabilistic Correlation Clustering and Image Partitioning Using Perturbed Multicuts, Inference on extremal dependence in the domain of attraction of a structured Hüsler-Reiss distribution motivated by a Markov tree with latent variables, Assignment Flows, Deep Variational Inference, Statistical mechanics of complex neural systems and high dimensional data, On one-step replica symmetry breaking in the Edwards–Anderson spin glass model, Loop expansion around the Bethe approximation through theM-layer construction, Correctness of sequential Monte Carlo inference for probabilistic programming languages, On the truncated Hausdorff moment problem under Sobolev regularity conditions, Simultaneous inference of periods and period-luminosity relations for Mira variable stars, Expectation Propagation with Factorizing Distributions: A Gaussian Approximation and Performance Results for Simple Models, The IBMAP approach for Markov network structure learning, A maximum likelihood algorithm for the estimation and renormalization of exponential densities, Variational Inference for Heteroscedastic Semiparametric Regression, Optical Flow, On entropy minimization and convergence, Robustness to Dependency in Portfolio Optimization Using Overlapping Marginals, Stein's method for stationary distributions of Markov chains and application to Ising models, New closed-form bounds on the partition function, Variable neighborhood search for graphical model energy minimization, Estimating the rate constant from biosensor data via an adaptive variational Bayesian approach, Link sign prediction by variational Bayesian probabilistic matrix factorization with Student-\(t\) prior, Model-based clustering of time-evolving networks through temporal exponential-family random graph models, Extension complexity of the correlation polytope, The Bayesian update: variational formulations and gradient flows, Syntactic stochastic processes: definitions, models, and related inference problems, Sum-product graphical models, Discriminator feature-based inference by recycling the discriminator of GANs, MAP inference via \(\ell_2\)-sphere linear program reformulation, A Gaussian graphical model approach to climate networks, Particle-based energetic variational inference, Relaxing Nonconvex Quadratic Functions by Multiple Adaptive Diagonal Perturbations, Updating variational Bayes: fast sequential posterior inference, The recursive variational Gaussian approximation (R-VGA), Estimating parameters of a directed weighted graph model with beta-distributed edge-weights, A Theoretical Examination of Diffusive Molecular Dynamics, Loss function, unbiasedness, and optimality of Gaussian graphical model selection, Learning hidden Markov models from aggregate observations, Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems, Bayesian model-based clustering for longitudinal ordinal data, Multiscale sparse microcanonical models, Analysis of generalized Bregman surrogate algorithms for nonsmooth nonconvex statistical learning, A Bayesian hierarchical score for structure learning from related data sets, A Bayesian nonparametric approach to super-resolution single-molecule localization, Comment: Variational autoencoders as empirical Bayes, Topology discovery of sparse random graphs with few participants, Mapping interstellar dust with Gaussian processes, Contention resolution, matrix scaling and fair allocation, Learning algorithm of Boltzmann machine based on spatial Monte Carlo integration method, A mathematical theory of communication, A variational inference for the Lévy adaptive regression with multiple kernels, On model selection consistency of regularized M-estimators, An Overview of Computational Sparse Models and Their Applications in Artificial Intelligence, Affine-mapping based variational ensemble Kalman filter, COAL: a generic modelling and prototyping framework for convex optimization problems of variational image analysis, Unnamed Item, Variational inference with vine copulas: an efficient approach for Bayesian computer model calibration, Energy distribution view for monotonic dual decomposition, Renewal model for dependent binary sequences, New limits of treewidth-based tractability in optimization, Topic model for graph mining based on hierarchical Dirichlet process, A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors, Inference of Population Structure from Ancient DNA, Monte Carlo Approximation of Bayes Factors via Mixing With Surrogate Distributions, Polynomial-time approximation algorithms for the antiferromagnetic Ising model on line graphs, Polynomial-time algorithms for multimarginal optimal transport problems with structure, Activity propagation in systems of linear inequalities and its relation to block-coordinate descent in linear programs, Super-reparametrizations of weighted CSPs: properties and optimization perspective, Longitudinal network models and permutation‐uniform Markov chains, Reducing bias due to misclassified exposures using instrumental variables, The Convergence of Markov Chain Monte Carlo Methods: From the Metropolis Method to Hamiltonian Monte Carlo, Hypothesis testing in adaptively sampled data: ART to maximize power beyond \textit{iid }sampling, Priors in Bayesian Deep Learning: A Review, Frequentist Model Averaging for Undirected Gaussian Graphical Models, Virtual pairwise consistency in cost function networks, Parameter estimation in a 3‐parameter p‐star random graph model, Limits of multi-relational graphs, Convergence of estimative density: criterion for model complexity and sample size, Black Box Variational Bayesian Model Averaging, The TAP free energy for high-dimensional linear regression, Near-Optimal Learning of Tree-Structured Distributions by Chow and Liu, Hedonic pricing modelling with unstructured predictors: an application to Italian fashion industry, Bayesian sparse spiked covariance model with a continuous matrix shrinkage prior, Energy-Based Models with Applications to Speech and Language Processing, A Friendly Tutorial on Mean-Field Spin Glass Techniques for Non-Physicists, Total positivity in multivariate extremes, Tutorial on Amortized Optimization, Reduced-order model-based variational inference with normalizing flows for Bayesian elliptic inverse problems, Variational Bayes estimation of hierarchical Dirichlet-multinomial mixtures for text clustering, The right complexity measure in locally private estimation: it is not the Fisher information, Semi-supervised invertible neural operators for Bayesian inverse problems, Probabilistic Registration for Gaussian Process Three-Dimensional Shape Modelling in the Presence of Extensive Missing Data, Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization, A Nonlocal Graph-PDE and Higher-Order Geometric Integration for Image Labeling, Minimizing oracle-structured composite functions, Locally Sparse Function-on-Function Regression, Optimal nonparametric testing of missing completely at random and its connections to compatibility, Scalable Bayesian computation for crossed and nested hierarchical models, Asymptotics in directed exponential random graph models with an increasing bi-degree sequence, Belief propagation: accurate marginals or accurate partition function—where is the difference?, Dynamic Sampling from Graphical Models, A Spectral Estimation Framework for Phase Retrieval via Bregman Divergence Minimization, Learning physical properties of anomalous random walks using graph neural networks, Second-order extended particle filter with exponential family observation model, The Complexity of General-Valued CSPs, Variational Inference for Latent Space Models for Dynamic Networks, Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields, Sequential change-point detection in high-dimensional Gaussian graphical models, Ising model selection using ℓ ₁-regularized linear regression: a statistical mechanics analysis*, Mean-field inference methods for neural networks, EXPLOITING SYNTACTIC, SEMANTIC, AND LEXICAL REGULARITIES IN LANGUAGE MODELING VIA DIRECTED MARKOV RANDOM FIELDS, Graphical Models and Message-Passing Algorithms: Some Introductory Lectures, Unnamed Item, Unnamed Item, Variational Bayesian inference for association over phylogenetic trees for microorganisms, Unnamed Item, Unnamed Item, Bayesian Brains and the Rényi Divergence, On the Complexity of Constrained Determinantal Point Processes, Data Analytics on Graphs Part I: Graphs and Spectra on Graphs, Dimension of Marginals of Kronecker Product Models, A Multiclass Classification Method Based on Decoding of Binary Classifiers, Bayesian Approaches to Shrinkage and Sparse Estimation, Unnamed Item, Unnamed Item, Unnamed Item, Unnamed Item, Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning, Convergence and Correctness of Max-Product Belief Propagation for Linear Programming, Active Mean Fields for Probabilistic Image Segmentation: Connections with Chan--Vese and Rudin--Osher--Fatemi Models, A Hilbert Space Embedding for Distributions, Variational Gaussian approximation for Poisson data, Improved High Dimensional Discrete Bayesian Network Inference using Triplet Region Construction, Implementing a Bayes Filter in a Neural Circuit: The Case of Unknown Stimulus Dynamics, Direct Learning of Sparse Changes in Markov Networks by Density Ratio Estimation, The Complexity of Valued CSPs, Bayesian adaptation of chaos representations using variational inference and sampling on geodesics, Inferring Sparse Preference Lists from Partial Information, Spatio-temporal Data Mining for Climate Data: Advances, Challenges, and Opportunities, Fast and Accurate Binary Response Mixed Model Analysis via Expectation Propagation, Gaussian Approximations for Probability Measures on $R^d$, A simple analytical description of the non-stationary dynamics in Ising spin systems, Spectral bounds for the Ising ferromagnet on an arbitrary given graph, LP Formulations for Polynomial Optimization Problems, On expectation propagation for generalised, linear and mixed models, Quantifying Information Conveyed by Large Neuronal Populations, Active Inference, Belief Propagation, and the Bethe Approximation, Unnamed Item, Unnamed Item, Unnamed Item, Nishimori meets Bethe: a spectral method for node classification in sparse weighted graphs, Uncertainty Quantification for Markov Random Fields, A Multiscale Community Blockmodel for Network Exploration, Accuracy of variational estimates for random graph mixture models, Physics-Constrained, Data-Driven Discovery of Coarse-Grained Dynamics, Empirical comparison study of approximate methods for structure selection in binary graphical models, Structure Learning of Contextual Markov Networks using Marginal Pseudo‐likelihood, Maximum likelihood estimation for discrete exponential families and random graphs, Unnamed Item, Unnamed Item, Unnamed Item, Unnamed Item, Unnamed Item, Conjugate Gamma Markov Random Fields for Modelling Nonstationary Sources, Image Labeling Based on Graphical Models Using Wasserstein Messages and Geometric Assignment, Correlated Random Measures, Structure learning in inverse Ising problems using ℓ ₂-regularized linear estimator, Review of statistical network analysis: models, algorithms, and software, A Model-Based Embedding Technique for Segmenting Customers, Bayesian consensus clustering in multiplex networks, EMVS: The EM Approach to Bayesian Variable Selection, Beyond Prediction: A Framework for Inference With Variational Approximations in Mixture Models, Exact or approximate inference in graphical models: why the choice is dictated by the treewidth, and how variable elimination can be exploited, Solving LP Relaxations of Some NP-Hard Problems Is As Hard As Solving Any Linear Program, Diffusion methods for classification with pairwise relationships, Statistical and Computational Guarantees for the Baum-Welch Algorithm, Deterministic counting of graph colourings using sequences of subgraphs, Worst-Case Expected Shortfall with Univariate and Bivariate Marginals, Frequentist Consistency of Variational Bayes, Unnamed Item, An EM Algorithm for Capsule Regression, Whence the Expected Free Energy?, One- versus multi-component regular variation and extremes of Markov trees, A Simple Algorithm for Sampling Colorings of $G(n,d/n)$ Up to The Gibbs Uniqueness Threshold, High-temperature expansions and message passing algorithms, Streamlining variational inference for constraint satisfaction problems, Gauging variational inference, The committee machine: computational to statistical gaps in learning a two-layers neural network, Learning performance in inverse Ising problems with sparse teacher couplings, The Power of Linear Programming for General-Valued CSPs, Approximate survey propagation for statistical inference, A Sufficient Statistic for Influence in Structured Multiagent Environments, Unnamed Item, Unnamed Item, Unnamed Item, Gauges, loops, and polynomials for partition functions of graphical models, Unnamed Item, Unnamed Item, Unnamed Item, Unnamed Item, Low-Density Parity-Check Codes Achieve List-Decoding Capacity, Gaussian Belief Propagation Solvers for Nonsymmetric Systems of Linear Equations, Learning to Approximate Industrial Problems by Operations Research Classic Problems