TG-CSR: Theoretically-Grounded Commonsense Reasoning Benchmark
DOI10.5281/zenodo.8279823Zenodo8279823MaRDI QIDQ6720457FDOQ6720457
Dataset published at Zenodo repository.
Ke Shen, Mayank Kejriwal, Henrique Santos, D. L. McGuinness, Alice M. Mulvehill
Publication date: 25 January 2023
Copyright license: Creative Commons Attribution 4.0 International
Achieving machine common sense has been a longstanding problem within Artificial Intelligence. Thus far, benchmark data sets that are grounded in a theory of common sense and can be used to conduct rigorous, semantic evaluations of common sense reasoning (CSR) systems have been lacking. One expectation of the AI community is that neuro-symbolic reasoners can help bridge this gap towards more dependable systems with common sense. We propose a novel benchmark, called Theoretically Grounded common sense Reasoning (TG-CSR), modeled as a set of question-answering instances, with each instance grounded in a semantic category of common sense, such as space, time, and emotions. The benchmark is few-shot i.e., only a few training and validation examples are provided in the public release to avoid the possibility of overfitting. Results from recent evaluations suggest that TG-CSR is challenging even for state-of-the-art statistical models. Due to its semantic rigor, this benchmark can be used to evaluate the common sense reasoning capabilities of neuro-symbolic systems.
This page was built for dataset: TG-CSR: Theoretically-Grounded Commonsense Reasoning Benchmark
