Autonomous GenAI Agents for Legacy-to-Cloud ETL Modernization

Vasudevan Ananthakrishnan (), Shemeer Sulaiman Kunju () and Radhakrishnan Pachyappan ()

Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, 2024, vol. 1, issue 1, 274-290

Abstract: The modernization of Extract, Transform, Load (ETL) processes from legacy systems to cloud-native architectures is critical for enhancing scalability, agility, and cost-efficiency in enterprise data management. Traditional manual modernization approaches, however, are time-intensive, error-prone, and require specialized expertise. This research introduces a novel framework leveraging autonomous Generative AI (GenAI) agents to automate the end-to-end legacy-to-cloud ETL modernization. The proposed agents autonomously analyze legacy ETL logic (e.g., SQL scripts, COBOL jobs), redesign pipelines using cloud-native services (e.g., AWS Glue, Azure Data Factory), generate optimized transformation code, validate data integrity, and deploy modularized workflows. Evaluations across real-world financial and healthcare datasets demonstrate a 70% reduction in migration time, 40% lower operational costs, and 99.5% schema consistency compared to manual methods. The framework also enables continuous optimization via adaptive learning from runtime metrics. This work pioneers AI-driven automation for legacy system modernization, significantly accelerating cloud adoption while minimizing risks.

Keywords: ETL Modernization; Generative AI Agents; Legacy System Migration; Cloud-Native Pipelines; Autonomous Data Engineering (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://newjaigs.com/index.php/JAIGS/article/view/377 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:das:njaigs:v:1:y:2024:i:1:p:274-290:id:377

Access Statistics for this article

Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023 is currently edited by Justyna Żywiołek

More articles in Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023 from Open Knowledge
Bibliographic data for series maintained by Open Knowledge ().