Hyperlink regression via Bregman divergence

DOI10.1016/J.NEUNET.2020.03.026MaRDI QIDQ1980417zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Akifumi Okuno, Hidetoshi Shimodaira

Publication date 8 September 2021

Published in Neural Networks (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1908.02573

zbMATH Keywords

Bregman divergencegraph embeddinghypernetworkneural network

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Generalized linear models (logistic models) (62J12) Graph theory (including graph drawing) in computer science (68R10) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: A collection of $U:(inmathbbN)$ data vectors is called a $U$-tuple, and the association strength among the vectors of a tuple is termed as the emph{hyperlink weight}, that is assumed to be symmetric with respect to permutation of the entries in the index. We herein propose Bregman hyperlink regression (BHLR), which learns a user-specified symmetric similarity function such that it predicts the tuple's hyperlink weight from data vectors stored in the $U$-tuple. BHLR is a simple and general framework for hyper-relational learning, that minimizes Bregman-divergence (BD) between the hyperlink weights and estimated similarities defined for the corresponding tuples; BHLR encompasses various existing methods, such as logistic regression ($U=1$), Poisson regression ($U=1$), link prediction ($U=2$), and those for representation learning, such as graph embedding ($U=2$), matrix factorization ($U=2$), tensor factorization ($Ugeq2$), and their variants equipped with arbitrary BD. Nonlinear functions (e.g., neural networks), can be employed for the similarity functions. However, there are theoretical challenges such that some of different tuples of BHLR may share data vectors therein, unlike the i.i.d. setting of classical regression. We address these theoretical issues, and proved that BHLR equipped with arbitrary BD and $UinmathbbN$ is (P-1) statistically consistent, that is, it asymptotically recovers the underlying true conditional expectation of hyperlink weights given data vectors, and (P-2) computationally tractable, that is, it is efficiently computed by stochastic optimization algorithms using a novel generalized minibatch sampling procedure for hyper-relational data. Consequently, theoretical guarantees for BHLR including several existing methods, that have been examined experimentally, are provided in a unified manner.

Recommendations

• New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation
• 10.1162/jmlr.2003.3.4-5.679
• Structure learning for relational logistic regression: an ensemble approach
• Multi-relational learning via hierarchical nonparametric Bayesian collective matrix factorization
• Learning theory for distribution regression

Cites work

• A Simple Proof of 1 + 1 2 2 + 1 3 2 + ⋯ = π 2 6 and Related Identities
• Adaptive subgradient methods for online learning and stochastic optimization
• Bregman Voronoi diagrams
• Clustering with Bregman divergences.
• Cross-validation of matching correlation analysis by resampling matching weights
• Global and Asymptotic Convergence Rate Estimates for a Class of Projected Gradient Processes
• scientific article; zbMATH DE number 47948 (Why is no real title available?)
• scientific article; zbMATH DE number 1382772 (Why is no real title available?)
• scientific article; zbMATH DE number 3296905 (Why is no real title available?)
• scientific article; zbMATH DE number 964896 (Why is no real title available?)
• Hypernetworks in the Science of Complex Systems
• Independent Component Analysis and Blind Signal Separation
• Kernel methods and machine learning
• Metric Learning
• New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation
• Pattern recognition and machine learning.
• Penalized Bregman divergence for large-dimensional regression and classification
• Random Dot Product Graph Models for Social Networks
• Regression analysis of count data
• Robust and efficient estimation by minimising a density power divergence
• Robust and sparse regression in generalized linear model by stochastic optimization
• Statistical inference on random dot product graphs: a survey
• Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
• Tensor Decompositions and Applications
• Uniform Convergence in Probability and Stochastic Equicontinuity
• Universally consistent vertex classification for latent positions graphs

Cited in

(1)

• Dependence of variance on covariate design in nonparametric link regression

Describes a project that uses

Uses Software

• PRMLT
• AdaGrad
• LINE
• word2vec
• SciPy
• Statsmodels
• Adam

This page was built for publication: Hyperlink regression via Bregman divergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1980417)