Refining neural network predictions using background knowledge

DOI10.1007/S10994-023-06310-3MaRDI QIDQ6176232zbMATH OpenFDO

Authors Alessandro Daniele, Emile van Krieken, Luciano Serafini, Frank van Harmelen

Publication date 22 August 2023

Published in Machine Learning (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2206.04976

zbMATH Keywords

optimizationfuzzy logicneurosymbolic AI

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) (68T20) Reasoning under uncertainty in the context of artificial intelligence (68T37)

Abstract: Recent work has shown logical background knowledge can be used in learning systems to compensate for a lack of labeled training data. Many methods work by creating a loss function that encodes this knowledge. However, often the logic is discarded after training, even if it is still useful at test time. Instead, we ensure neural network predictions satisfy the knowledge by refining the predictions with an extra computation step. We introduce differentiable refinement functions that find a corrected prediction close to the original prediction. We study how to effectively and efficiently compute these refinement functions. Using a new algorithm called Iterative Local Refinement (ILR), we combine refinement functions to find refined predictions for logical formulas of any complexity. ILR finds refinements on complex SAT formulas in significantly fewer iterations and frequently finds solutions where gradient descent can not. Finally, ILR produces competitive results in the MNIST addition task.

Recommendations

• Refining numerical constants in first order logic theories
• Semi-supervised learning using differentiable reasoning
• scientific article; zbMATH DE number 2009985
• scientific article; zbMATH DE number 1875386
• High-order networks that learn to satisfy logic constraints

Cites work

• scientific article; zbMATH DE number 3942105 (Why is no real title available?)
• scientific article; zbMATH DE number 1986059 (Why is no real title available?)
• Analyzing differentiable fuzzy logic operators
• Cone monotonicity: structure theorem, properties, and comparisons to other notions of monotonicity
• Fuzzy implications
• Inequalities: theory of majorization and its applications
• Logic tensor networks
• Multi-label classification neural networks with hard logical constraints
• SATLIB: An online resource for research on SAT
• Schur-concave triangular norms: Characterization and application in pFCSP
• Semantic-based regularization for learning and inference
• Subgradient Criteria for Monotonicity, The Lipschitz Condition, and Convexity
• Triangular norms. Position paper II: General constructions and parameterized families

This page was built for publication: Refining neural network predictions using background knowledge

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6176232)