Purifying Data by Machine Learning with Certainty Levels

Shlomi Dolev () and Guy Leshem ()
Additional contact information
Shlomi Dolev: Ben-Gurion University of the Negev
Guy Leshem: Ashkelon Academic College

A chapter in Data Analysis and Optimization, 2023, pp 89-102 from Springer

Abstract: Abstract For autonomic computing, self-managing systems, and decision-making under uncertainty and faults, in many cases we are using machine learning models and combine them to solve any problem. This models uses a data-set, or a set of data-items, and data-item is a vector of feature values and a classification. in many cases these data sets includes outlier and/or misleading data items that were created by input device malfunctions, or were maliciously inserted to lead the machine learning to wrong conclusions. A reliable machine learning model must be able to handle a corrupted data-set, otherwise, a malfunctioning input device that corrupts a portion of the data-set, or malicious adversary may lead to inaccurate classifications. Therefore, the challenge is to find an effective methods to evaluate and increase the certainty level of the learning process as much as possible. This work introduces the use of a certainty level measure to obtain better classification capability in the presence of corrupted or malicious data items. Assuming we know the data distribution, e.g., is a normal distribution (which is a reasonable assumption in a large amount of data items) and/or a known upper bound on the given number of corrupted data items, our techniques define a certainty level for classifications. Another approach that will be presented in this work suggests enhancing the random forest techniques (the original model was developed by Leo Breiman) to cope with corrupted data items by augmenting the certainty level for the classification obtained in each leaf in the forest. This method is of independent interest, that of significantly improving the classification of the random forest machine learning technique in less severe settings.

Keywords: Data corruption; PAC learning; Machine learning; Certainty level (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:spochp:978-3-031-31654-8_6

Ordering information: This item can be ordered from
http://www.springer.com/9783031316548

DOI: 10.1007/978-3-031-31654-8_6

Access Statistics for this chapter

More chapters in Springer Optimization and Its Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().