EVALUATION OF CLASSIFICATION ALGORITHMS USING MCDM AND RANK CORRELATION

Gang Kou, Yanqun Lu, Yi Peng () and Yong Shi
Additional contact information
Gang Kou: School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, 610054, P. R. China
Yanqun Lu: School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, 610054, P. R. China
Yi Peng: School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, 610054, P. R. China
Yong Shi: College of Information Science & Technology, University of Nebraska at Omaha, Omaha, NE 68182, USA;

International Journal of Information Technology & Decision Making (IJITDM), 2012, vol. 11, issue 01, 197-225

Abstract: Classification algorithm selection is an important issue in many disciplines. Since it normally involves more than one criterion, the task of algorithm selection can be modeled as multiple criteria decision making (MCDM) problems. Different MCDM methods evaluate classifiers from different aspects and thus they may produce divergent rankings of classifiers. The goal of this paper is to propose an approach to resolve disagreements among MCDM methods based on Spearman's rank correlation coefficient. Five MCDM methods are examined using 17 classification algorithms and 10 performance criteria over 11 public-domain binary classification datasets in the experimental study. The rankings of classifiers are quite different at first. After applying the proposed approach, the differences among MCDM rankings are largely reduced. The experimental results prove that the proposed approach can resolve conflicting MCDM rankings and reach an agreement among different MCDM methods.

Keywords: Multi-criteria decision making (MCDM); classification; Spearman's rank correlation coefficient; TOPSIS; ELECTRE; grey relational analysis; VIKOR; PROMETHEE (search for similar items in EconPapers)
Date: 2012
References: View complete reference list from CitEc
Citations: View citations in EconPapers (267)

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0219622012500095
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:ijitdm:v:11:y:2012:i:01:n:s0219622012500095

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0219622012500095

Access Statistics for this article

International Journal of Information Technology & Decision Making (IJITDM) is currently edited by Yong Shi

More articles in International Journal of Information Technology & Decision Making (IJITDM) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().