Data Integration Process Automation Using Machine Learning: Issues and Solution

Kartick Chandra Mondal () and Swati Saha ()
Additional contact information
Kartick Chandra Mondal: Jadavpur University, Department of Information Technology
Swati Saha: Tata Consultancy Services

A chapter in Machine Learning for Data Science Handbook, 2023, pp 39-54 from Springer

Abstract: Abstract In today’s data-driven world, real-time analysis of enterprise data plays an important role in the organization to take strategic decisions and improve business operations. The availability of data in real time and analyzing those data instantly are becoming a challenge for most organizations. Outdated data do not add any value to an organization. The company needs a reliable, minute-to-minute information to improve operational efficiency and make better proactive business decisions. Typically, running a data warehouse in an enterprise requires coordination of many operations across multiple teams. Also, a lot of manual intervention is required, which is error-prone. Executing all related steps in correct sequences under correct conditions can be a challenge. The automated data integration, specifically, ETL (Extract-Transform-Load) process, is the only solution to address all these problems. Improving ETL process system data flows can provide a better return on your business investment. Since, data across multiple systems are integrated into data warehouse (DWH). There can be quality issues of integrated data that can generate inaccurate analytic. Hence, data need to be pre-processed and optimized for the business intelligence process. Automated data integration, specifically the ETL process, can address the issues of traditional data warehouse related to availability and quality of data. Here, the solution approach of the automated ETL process is explained, which supports continuous integration. It also describes how machine learning can be leveraged in the ETL process so that the quality and availability of data not ever have been compromised.

Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-24628-9_3

Ordering information: This item can be ordered from
http://www.springer.com/9783031246289

DOI: 10.1007/978-3-031-24628-9_3

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().