Permutation Tests for Metaheuristic Algorithms

Mahamed G. H. Omran, Maurice Clerc, Fatme Ghaddar, Ahmad Aldabagh and Omar Tawfik
Additional contact information
Mahamed G. H. Omran: Centre for Applied Mathematics and Bioinformatics, and Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait
Maurice Clerc: Independent Consultant, France
Fatme Ghaddar: Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait
Ahmad Aldabagh: Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait
Omar Tawfik: Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait

Mathematics, 2022, vol. 10, issue 13, 1-15

Abstract: Many metaheuristic approaches are inherently stochastic. In order to compare such methods, statistical tests are needed. However, choosing an appropriate test is not trivial, given that each test has some assumptions about the distribution of the underlying data that must be true before it can be used. Permutation tests (P-Tests) are statistical tests with minimal number of assumptions. These tests are simple, intuitive and nonparametric. In this paper, we argue researchers in the field of metaheuristics to adopt P-Tests to compare their algorithms. We define two statistic tests and then present an algorithm that uses them to compute the p -value. The proposed process is used to compare 5 metaheuristic algorithms on 10 benchmark functions. The resulting p -values are compared with the p -values of two widely used statistical tests. The results show that the proposed P-test is generally consistent with the classical tests, but more conservative in few cases.

Keywords: metaheuristics; stochastic search; optimization; statistical tests; nonparametric tests; permutation tests (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/13/2219/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/13/2219/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:13:p:2219-:d:847282

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().