Ranking of Classification Algorithms in Terms of Mean–Standard Deviation Using A-TOPSIS

André G. C. Pacheco () and Renato A. Krohling ()
Additional contact information
André G. C. Pacheco: UFES - Federal University of Espirito Santo
Renato A. Krohling: UFES - Federal University of Espirito Santo

Annals of Data Science, 2018, vol. 5, issue 1, No 8, 93-110

Abstract: Abstract In classification problems when multiple algorithms are applied to different benchmarks a difficult issue arises, i.e., how can we rank the algorithms? In machine learning, it is common to run the algorithms several times and then a statistic is calculated in terms of means and standard deviations. In order to compare the performance of the algorithms, it is very common to employ statistical tests. However, these tests may also present limitations, since they consider only the means and not the standard deviations of the obtained results. In this paper, we present the so-called A-TOPSIS, based on Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), to solve the problem of ranking and comparing classification algorithms in terms of means and standard deviations. We use two case studies to illustrate the A-TOPSIS for ranking classification algorithms and the results show the suitability of A-TOPSIS to rank the algorithms. The presented approach can be applied to compare the performance of stochastic algorithms in machine learning. Lastly, to encourage researchers to use the A-TOPSIS for ranking algorithms, we also presented in this work an easy-to-use A-TOPSIS web framework.

Keywords: Ranking algorithms; Machine learning; Classification algorithms; Comparison of algorithms; TOPSIS; Statistical tests (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s40745-018-0136-5 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:aodasc:v:5:y:2018:i:1:d:10.1007_s40745-018-0136-5

Ordering information: This journal article can be ordered from
https://www.springer ... gement/journal/40745

DOI: 10.1007/s40745-018-0136-5

Access Statistics for this article

Annals of Data Science is currently edited by Yong Shi

More articles in Annals of Data Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().