Real-Time Data Quality Monitoring System for Data Cleansing

Cihan Varol and Henry Neumann
Additional contact information
Cihan Varol: Sam Houston State University, USA
Henry Neumann: Sam Houston State University, USA

International Journal of Business Intelligence Research (IJBIR), 2012, vol. 3, issue 1, 83-93

Abstract: To assist business intelligence companies dealing with data preparation problems, different approaches have been developed to handle the dirty data. However, these data cleansing approaches do not have real-time monitoring capabilities. Therefore, business intelligence companies and their clients are not able to predict the final outcome before running all business process. This yields an extra cost for the company if the data are highly corrupted. Therefore, to reduce cost for these types of businesses, the authors design a framework that monitors the quality attributes during the data cleansing process. Moreover, the system provides feedback to the user and allows the user to restructure the workflow based on quality attributes. The main concept of the framework is based on client-server architecture that uses multithreading to allow real-time monitoring of the process. A child thread is dedicated to run and another is dedicated to monitor the processes and give feedback to the user. The real-time monitoring system not only displays the cleansing process done on the data set, but also estimates the risk propagation probabilities in the data cleansing process. De-duplication elimination, address normalization, spelling correction for personal names, and non-ASCII character removal techniques are employed.

Date: 2012
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 4018/jbir.2012010106 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jbir00:v:3:y:2012:i:1:p:83-93

Access Statistics for this article

International Journal of Business Intelligence Research (IJBIR) is currently edited by Ana Azevedo

More articles in International Journal of Business Intelligence Research (IJBIR) from IGI Global
Bibliographic data for series maintained by Journal Editor ().