Assessing Algorithmic Performance by Frontier Analysis: A DEA Approach

Jose Humberto Ablanedo-Rosas and Cesar Rego
Additional contact information
Jose Humberto Ablanedo-Rosas: University of Texas at El Paso, El Paso, TX, USA
Cesar Rego: University of Mississippi, Oxford, MS, USA

International Journal of Applied Metaheuristic Computing (IJAMC), 2018, vol. 9, issue 1, 78-94

Abstract: In Combinatorial Optimization the evaluation of heuristic algorithms often requires the consideration of multiple performance metrics that are relevant for the application of interest. Traditional empirical analysis of algorithms relies on evaluating individual performance metrics where the overall assessment is conducted by subjective judgment without the support of rigorous scientific methods. The authors propose an analytical approach based on data envelopment analysis (DEA) to rank algorithms by their relative efficiency scores that result from combining multiple performance metrics. To evaluate their approach, they perform a pilot study examining the relative performance of ten surrogate constraint algorithms for different classes of the set covering problem. The analysis shows their DEA-based approach is highly effective, establishing a clear difference between the algorithms' performances at appropriate statistical significance levels, and in consequence providing useful insights into the selection of algorithms to address each class of instances. Their approach is general and can be used with all types of performance metrics and algorithms.

Date: 2018
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 018/ijamc.2018010106 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jamc00:v:9:y:2018:i:1:p:78-94

Access Statistics for this article

International Journal of Applied Metaheuristic Computing (IJAMC) is currently edited by Peng-Yeng Yin

More articles in International Journal of Applied Metaheuristic Computing (IJAMC) from IGI Global
Bibliographic data for series maintained by Journal Editor ().