An empirical analysis of terminological representation systems (Q1332849)

The paper studies and compares six terminological representation systems, which are intended to support the taxonomic representation of terminology and to provide reasoning services about it. The following systems are included in the study: BACK, CLASSIC, KRIS, LOOM, SB-ONE, and MESON. All of them are written in CommonLisp, and tested on a MacIvory. The empirical analysis performed on these systems is divided into two parts. The first part covers qualitative facts concerning system features and expressiveness, while the second part deals with performance issues. Five different types of experiments are carried out, namely: tests dealing with cases not covered by the common semantic framework of terminological representation systems, tests exploring the degree of the inferential completeness of the systems for polynomial inferences, tests exploring how the systems react on non-polynomial inferences, tests evaluating systems' performances under ``realistic'' conditions, and finally tests showing how the translation influences the performance. It is shown that the differences in expressiveness between these systems are larger than expected, even though all of them use a common semantic framework. As a result, serious problems can occur when knowledge bases are transported between the systems. The results concerning the runtime performance of these systems show that the structure of the knowledge base can have a significant impact on the performance, and that the runtime grows faster than linearly in all of the systems that have been evaluated. These results are complementary to the ones on the computational complexity of terminological reasoning.