Preliminary analysis on data quality for ML applications

Lorenz Kiebler, Nikolas Ulrich Moroff and Jens Jakob Jacobsen

A chapter in Changing Tides: The New Role of Resilience and Sustainability in Logistics and Supply Chain Management – Innovative Approaches for the Shift to a New Era, 2022, pp 207-236 from Hamburg University of Technology (TUHH), Institute of Business Logistics and General Management

Abstract: Purpose: This publication investigates preliminary data quality analyses to estimate the efforts and expected results of the use of data sets for ML solutions already in the data understanding phase of an implementation. Knowledge about the necessary data cleaning efforts and result qualities allows potentials to be estimated early in the process. Methodology: Through a literature research, characteristics of a time series as well as methods of data cleaning are analysed. Based on the results, a test environment is implemented in Python, enabling the evaluation of individual methods using sample data sets from the process industry and comparing them with different error analyses. Findings: The publication describes a detailed overview of data cleaning procedures and addresses a first Indication of a connection between the final achievable forecast quality and the degree of error of the original data set. Insights into the influence of the choice of preprocessing method on the achievable quality of the AI-based forecast can be concluded. Originality: Within the publication, the link between data characteristics in time series and preprocessing methods is established to draw conclusions in advance about the quality improvement to be expected from selected data cleaning methods and to provide decision support for the selection of the method.

Keywords: Artificial Intelligence; Blockchain (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/267187/1/hicl-2021-33-207.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:hiclch:267187

DOI: 10.15480/882.4693

Access Statistics for this chapter

More chapters in Chapters from the Proceedings of the Hamburg International Conference of Logistics (HICL) from Hamburg University of Technology (TUHH), Institute of Business Logistics and General Management
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().