Measuring the effectiveness and efficiency of simulation optimization metaheuristic algorithms

Benjamin G. Thengvall (), Shane N. Hall () and Michael P. Deskevich ()
Additional contact information
Benjamin G. Thengvall: OptTek Systems, Inc.
Shane N. Hall: Montana State University
Michael P. Deskevich: OptTek Systems, Inc.

Journal of Heuristics, 2025, vol. 31, issue 1, No 12, 21 pages

Abstract: Abstract Metaheuristic algorithms have proven capable as general-purpose algorithms for solving simulation optimization problems. Researchers and practitioners often compare different metaheuristic algorithms by examining one or more measures that are derived through empirical analysis. This paper presents a single measure that can be used to empirically compare different metaheuristic algorithms for optimization problems. This measure incorporates both the effectiveness and efficiency of the metaheuristic algorithm, which is especially important in simulation optimization applications because the number of simulation runs available to the analyst (i.e., the run budget) can vary significantly with each simulation study. Therefore, the trade-off between the effectiveness and efficiency of a metaheuristic algorithm must be examined. This single measure is especially useful for multi-objective optimization problems; however, determining this measure is non-trivial for two or more objective functions. Additional details for calculating this measure for multi-objective optimization problems are provided as well as a procedure for comparing two or more metaheuristic algorithms. Finally, computational results are presented and analyzed to compare the performance of metaheuristic algorithms using knapsack problems, pure binary integer programs, traveling salesman problems, and the average results obtained across a diverse set of optimization problems that include simulation and multi-objective optimization problems.

Keywords: Binary integer program; Efficiency and effectiveness of metaheuristics; Knapsack problem; Metaheuristic performance measures; Multi-objective optimization; Simulation optimization; Traveling salesman problem (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10732-025-09549-2 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joheur:v:31:y:2025:i:1:d:10.1007_s10732-025-09549-2

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10732

DOI: 10.1007/s10732-025-09549-2

Access Statistics for this article

Journal of Heuristics is currently edited by Manuel Laguna

More articles in Journal of Heuristics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().