Full-span log-linear model and fast learning algorithm

DOI10.1162/NECO_A_01496MaRDI QIDQ5081145zbMATH OpenOpenAlexWikidataFDO

Authors Kazuya Takabatake, Shotaro Akaho

Publication date 13 June 2022

Published in Neural Computation (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2202.08472

zbMATH Keywords

dual parametershighest-order Boltzmann machine

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: The full-span log-linear(FSLL) model introduced in this paper is considered an $n$-th order Boltzmann machine, where $n$ is the number of all variables in the target system. Let $X=(X_0,\mathrm{...},X_{n-1})$ be finite discrete random variables that can take $|X|=|X_0|\mathrm{...}|X_{n-1}|$ different values. The FSLL model has $|X|-1$ parameters and can represent arbitrary positive distributions of $X$. The FSLL model is a "highest-order" Boltzmann machine; nevertheless, we can compute the dual parameters of the model distribution, which plays important roles in exponential families, in $O(|X|log|X|)$ time. Furthermore, using properties of the dual parameters of the FSLL model, we can construct an efficient learning algorithm. The FSLL model is limited to small probabilistic models up to $|X|approx{2}^{25}$; however, in this problem domain, the FSLL model flexibly fits various true distributions underlying the training data without any hyperparameter tuning. The experiments presented that the FSLL successfully learned six training datasets such that $|X|=2^{20}$ within one minute with a laptop PC.

Recommendations

• Neural learning for distributions on categorical data
• Learning algorithm for Boltzmann machines using max-product algorithm and pseudo-likelihood
• A block successive lower-bound maximization algorithm for the maximum pseudo-likelihood estimation of fully visible Boltzmann machines
• scientific article; zbMATH DE number 3984350
• Consistency of Pseudolikelihood Estimation of Fully Visible Boltzmann Machines

Cites work

• scientific article; zbMATH DE number 1782878 (Why is no real title available?)
• scientific article; zbMATH DE number 5060482 (Why is no real title available?)
• Information and complexity in statistical modeling.
• Information geometry and its applications
• On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes
• Quasi-Newton Methods, Motivation and Theory
• Unified Matrix Treatment of the Fast Walsh-Hadamard Transform

Cited in

(1)

• \(\text{ALR}^n\): accelerated higher-order logistic regression

Describes a project that uses

Uses Software

• JSAT

This page was built for publication: Full-span log-linear model and fast learning algorithm

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5081145)