IMPROVING ADAPTABILITY OF DECISION TREE FOR MINING BIG DATA

Hang Yang () and Simon Fong ()
Additional contact information
Hang Yang: Department of Computer and Information Science, University of Macau, Av. Padre Toms Pereira Taipa, Macau, China
Simon Fong: Department of Computer and Information Science, University of Macau, Av. Padre Toms Pereira Taipa, Macau, China

New Mathematics and Natural Computation (NMNC), 2013, vol. 09, issue 01, 77-95

Abstract: Big data has become a popular research topic since the data explosion in the past decade. An efficient analytical methodology provides a way of discovering the potential value from big data. Sampling technique is unsuitable any more that the full data will tell the truths. To this end, the data mining algorithm shall be robust to imperfect data, which may lead to tree size explosion and detrimental accuracy problems. In this paper, we propose an incremental optimization mechanism to solve these problems. The mechanism is called Optimized Very Fast Decision Tree (OVFDT) that possesses an optimized node-splitting control mechanism. Accuracy, tree size and the learning time are the significant factors, which contribute to a much-improved algorithm in performance. Naturally a bigger tree size takes longer computation time. OVFDT is a pioneer model equipped with an incremental optimization mechanism that seeks for a balance between accuracy and tree size for data stream mining. OVFDT operates incrementally by a test-then-train approach. Two new methods of functional tree leaves are proposed to improve the accuracy that the tree model makes a prediction for a new data stream in the testing phase. The optimized node-splitting mechanism controls the tree model growth in the training phase. The experiment supports our claim that OVFDT is an optimal tree structure in both numeric and nominal datasets.

Keywords: Algorithms performance evaluation; data stream mining; decision tree (search for similar items in EconPapers)
Date: 2013
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S1793005713500063
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:nmncxx:v:09:y:2013:i:01:n:s1793005713500063

Ordering information: This journal article can be ordered from

DOI: 10.1142/S1793005713500063

Access Statistics for this article

New Mathematics and Natural Computation (NMNC) is currently edited by Paul P Wang

More articles in New Mathematics and Natural Computation (NMNC) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().