Effective BERT and GPT Integration for Ontology Development

Aleksandr M. Katyshev and Anton V. Anikin
Additional contact information
Aleksandr M. Katyshev: Volgograd State Technical University
Anton V. Anikin: Volgograd State Technical University

Economics. Information Technologies, 2026, vol. 53, issue 1, 144-152

Abstract: This paper addresses the challenge of automated ontology construction, particularly for morphologically rich languages like Russian, where existing tools such as Text2Onto and FRED show significant limitations. We introduce a novel hybrid methodology that synergistically integrates two powerful transformer-based models to build comprehensive ontological knowledge bases from Russian text corpora. The primary objective is to overcome the trade-off between precision and recall inherent in single-model approaches. Our proposed framework operates in a two-stage process. Initially, a Russian-adapted Bidirectional Encoder Representations from Transformers (BERT) model is employed for high-precision extraction of explicit knowledge. Leveraging its deep contextual understanding, BERT performs named entity recognition to identify candidate concepts and extracts a foundational set of semantic relationships through a sentence-pair classification approach. Subsequently, a fine-tuned Generative Pre-trained Transformer (GPT) model is utilized for knowledge enrichment and recall enhancement. GPT generates plausible hypotheses about unstated or implicit relationships between concepts, refines and verifies relations found by BERT, and resolves logical conflicts, thereby filling knowledge gaps. An empirical evaluation was conducted on a corpus of educational texts on web development to validate the method efficacy. The combined BERT+GPT approach demonstrated superior performance, achieving an F1-measure of 0.82, which significantly surpasses standalone BERT (0.80), FRED (0.62), and Text2Onto (0.52). This improvement is primarily attributed to a substantial increase in recall (0.81) while maintaining high precision (0.82). The practical application and utility of the generated ontologies are discussed in the context of their integration with knowledge management platforms like Stardog, enabling advanced semantic search, data enrichment, and logical inference capabilities.

Keywords: ontological knowledge base; knowledge graph; ontology learning; bert; gpt (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eul:ecoinf:v:53:y:2026:i:1:p:144-152

DOI: 10.52575/2687-0932-2026-53-1-144-152

Access Statistics for this article

Economics. Information Technologies is currently edited by Elena A. Stryabkova

More articles in Economics. Information Technologies from Belgorod State University
Bibliographic data for series maintained by Tatyana M. Khomenko ().