Consensus-based modeling using distributed feature construction with ILP
From MaRDI portal
Publication:1640573
DOI10.1007/S10994-017-5672-2zbMATH Open1458.68162arXiv1409.3446OpenAlexW2767061289MaRDI QIDQ1640573FDOQ1640573
Authors: Haimonti Dutta, Ashwin Srinivasan 
Publication date: 14 June 2018
Published in: Machine Learning (Search for Journal in Brave)
Abstract: A particularly successful role for Inductive Logic Programming (ILP) is as a tool for discovering useful relational features for subsequent use in a predictive model. Conceptually, the case for using ILP to construct relational features rests on treating these features as functions, the automated discovery of which necessarily requires some form of first-order learning. Practically, there are now several reports in the literature that suggest that augmenting any existing features with ILP-discovered relational features can substantially improve the predictive power of a model. While the approach is straightforward enough, much still needs to be done to scale it up to explore more fully the space of possible features that can be constructed by an ILP system. This is in principle, infinite and in practice, extremely large. Applications have been confined to heuristic or random selections from this space. In this paper, we address this computational difficulty by allowing features to be constructed in a distributed manner. That is, there is a network of computational units, each of which employs an ILP engine to construct some small number of features and then builds a (local) model. We then employ a consensus-based algorithm, in which neighboring nodes share information to update local models. For a category of models (those with convex loss functions), it can be shown that the algorithm will result in all nodes converging to a consensus model. In practice, it may be slow to achieve this convergence. Nevertheless, our results on synthetic and real datasets that suggests that in relatively short time the "best" node in the network reaches a model whose predictive accuracy is comparable to that obtained using more computational effort in a non-distributed setting (the best node is identified as the one whose weights converge first).
Full work available at URL: https://arxiv.org/abs/1409.3446
Recommendations
Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Learning and adaptive systems in artificial intelligence (68T05) Logic programming (68N17)
Cites Work
- TopLog: ILP Using a Logic Program Declarative Bias
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Title not available (Why is that?)
- Title not available (Why is that?)
- Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
- Title not available (Why is that?)
- Title not available (Why is that?)
- Distributed asynchronous deterministic and stochastic gradient optimization algorithms
- Title not available (Why is that?)
- Learning Boolean functions in an infinite attribute space
- Impossibility of distributed consensus with one faulty process
- Foundations of inductive logic programming
- Gossip algorithms
- Parallel Gradient Distribution in Unconstrained Optimization
- Solving feature subset selection problem by a parallel scatter search
- Massively parallel feature selection: an approach based on variance preservation
- Integrating naïve Bayes and FOIL
- Optimal distributed online prediction using mini-batches
- Word Sense Disambiguation Using Inductive Logic Programming
- An introduction to measure theory
- Learning to parse natural language with maximum entropy models
- Geographic Gossip: Efficient Averaging for Sensor Networks
- Order-Optimal Consensus Through Randomized Path Averaging
- Randomised restarted search in ILP
- An investigation into feature construction to assist word sense disambiguation
- An empirical study of on-line models for relational data streams
- A reliable effective terascale linear learning system
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- What kinds of relational features are useful for statistical learning?
- Parameter screening and optimisation for ILP using designed experiments
- Using ILP to Construct Features for Information Extraction from Semi-structured Text
Uses Software
This page was built for publication: Consensus-based modeling using distributed feature construction with ILP
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1640573)
