Artificial Intelligence Driven Trend Forecasting: Integrating BERT Topic Modelling and Generative Artificial Intelligence for Semantic Insights

Deepak Kumar and Marion Weissenberger-Eibl

EconStor Conference Papers from ZBW - Leibniz Information Centre for Economics

Abstract: In the fast-paced realm of technological evolution, accurately forecasting emerging trends is critical for both academic inquiry and industry application. Traditional trend analysis methodologies, while valuable, struggle to efficiently process and interpret the vast datasets of today's information age. This paper introduces a novel approach that synergizes Generative AI and Bidirectional Encoder Representations from Transformers (BERT) for semantic insights and trend forecasting, leveraging the power of Retrieval-Augmented Generation (RAG) and the analytical prowess of BERT topic modeling. By automating the analysis of extensive datasets from publications and patents, the presented methodology not only expedites the discovery of emergent trends but also enhances the precision of these findings by generating a short summary for found emergent trends. For validation, three technologies - reinforcement learning, quantum machine learning, and Cryptocurrencies - were analysed prior to their first appearance in the Gartner Hype Cycle. Research highlights the integration of advanced AI techniques in trend forecasting, providing a scalable and accurate tool for strategic planning and innovation management. Results demonstrated a significant correlation between model's predictions and the technologies' appearances in the Hype Cycle, underscoring the potential of this methodology in anticipating technological shifts across various sectors

Keywords: BERT; Topic modelling; RAG; Gartner Hype Cycle; LLM; BERTopic (search for similar items in EconPapers)
Date: 2024
New Economics Papers: this item is included in nep-ain, nep-big, nep-cmp and nep-ino
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/300545/1/A ... rend-Forecasting.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:esconf:300545

Access Statistics for this paper

More papers in EconStor Conference Papers from ZBW - Leibniz Information Centre for Economics Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().