Rule Extraction from Decision Trees Ensembles: New Algorithms Based on Heuristic Search and Sparse Group Lasso Methods

Morteza Mashayekhi and Robin Gras
Additional contact information
Morteza Mashayekhi: School of Computer Science, University of Windsor, 401 Sunset Ave., Windsor, ON N9B3P4, Canada
Robin Gras: School of Computer Science, University of Windsor, 401 Sunset Ave., Windsor, ON N9B3P4, Canada

International Journal of Information Technology & Decision Making (IJITDM), 2017, vol. 16, issue 06, 1707-1727

Abstract: Decision trees are examples of easily interpretable models whose predictive accuracy is normally low. In comparison, decision tree ensembles (DTEs) such as random forest (RF) exhibit high predictive accuracy while being regarded as black-box models. We propose three new rule extraction algorithms from DTEs. The RF+DHC method, a hill climbing method with downhill moves (DHC), is used to search for a rule set that decreases the number of rules dramatically. In the RF+SGL and RF+MSGL methods, the sparse group lasso (SGL) method, and the multiclass SGL (MSGL) method are employed respectively to find a sparse weight vector corresponding to the rules generated by RF. Experimental results with 24 data sets show that the proposed methods outperform similar state-of-the-art methods, in terms of human comprehensibility, by greatly reducing the number of rules and limiting the number of antecedents in the retained rules, while preserving the same level of accuracy.

Keywords: Rule extraction (RE); decision tree ensemble (DTE); hill climbing (HC); sparse group lasso (SGL); multiclass sparse group lasso (MSGL) (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0219622017500055
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:ijitdm:v:16:y:2017:i:06:n:s0219622017500055

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0219622017500055

Access Statistics for this article

International Journal of Information Technology & Decision Making (IJITDM) is currently edited by Yong Shi

More articles in International Journal of Information Technology & Decision Making (IJITDM) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().