Abstract
Ontology development involves a top-down approach where ontology engineers and domain experts collaboratively define and evaluate ontological elements and axioms. Translating ontology axioms into natural language can significantly aid in ontology evaluation by making the content more understandable to subject matter experts who may lack a background in knowledge engineering. In this preliminary study, we investigate the potential of large language models (LLMs) in axiom translation from ontologies to facilitate ontology evaluation. We utilize Llama 3 to translate 1,192 ontology axioms across 19 distinct axiom types from five published ontologies. Results show that 163 (13.67%) of the Llama 3 translation of the axiom are accurately represented, 268 (22.48%) are not accurately represented, and 761 (63.84%) are partially accurate. Our manual evaluation of the Llama 3 translation indicates some competency in producing hierarchical natural language equivalents while revealing some limitations when translating complex axioms. Nonetheless, there are opportunities to improve the results with few-shot training or using LLMs to provide support in knowledge engineering for ontologies.
Original language | English (US) |
---|---|
Journal | CEUR Workshop Proceedings |
Volume | 3853 |
State | Published - 2024 |
Event | Joint of the 2nd Workshop on Knowledge Base Construction from Pre-Trained Language Models and the 3rd Challenge on Language Models for Knowledge Base Construction, KBC-LM-LM-KBC 2024 - Baltimore, United States Duration: Nov 12 2024 → … |
ASJC Scopus subject areas
- General Computer Science