Review of the Research Landscape of Multi-Criteria Evaluation and Benchmarking Processes for Many-Objective Optimization Methods: Coherent Taxonomy, Challenges and Recommended Solution

R. T. Mohammed, R. Yaakob, A. A. Zaidan, N. M. Sharef, R. H. Abdullah, B. B. Zaidan and K. A. Dawood
Additional contact information
R. T. Mohammed: #x2020;Faculty of Computer Science and Information Technology, Universiti Putra Malaysia (UPM), Selangor, Malaysia
R. Yaakob: #x2020;Faculty of Computer Science and Information Technology, Universiti Putra Malaysia (UPM), Selangor, Malaysia
A. A. Zaidan: Computing Department, Universiti Pendidikan Sultan Idris (UPSI), Tanjung Malim, Malaysia
N. M. Sharef: #x2020;Faculty of Computer Science and Information Technology, Universiti Putra Malaysia (UPM), Selangor, Malaysia
R. H. Abdullah: #x2020;Faculty of Computer Science and Information Technology, Universiti Putra Malaysia (UPM), Selangor, Malaysia
B. B. Zaidan: Computing Department, Universiti Pendidikan Sultan Idris (UPSI), Tanjung Malim, Malaysia
K. A. Dawood: #x2020;Faculty of Computer Science and Information Technology, Universiti Putra Malaysia (UPM), Selangor, Malaysia

International Journal of Information Technology & Decision Making (IJITDM), 2020, vol. 19, issue 06, 1619-1693

Abstract: Evaluation and benchmarking of many-objective optimization (MaOO) methods are complicated. The rapid development of new optimization algorithms for solving problems with many objectives has increased the necessity of developing performance indicators or metrics for evaluating the performance quality and comparing the competing optimization algorithms fairly. Further investigations are required to highlight the limitations of how criteria/metrics are determined and the consistency of the procedures with the evaluation and benchmarking processes of MaOO. A review is conducted in this study to map the research landscape of multi-criteria evaluation and benchmarking processes for MaOO into a coherent taxonomy. Then contentious and challenging issues related to evaluation are highlighted, and the performance of optimization algorithms for MaOO is benchmarked. The methodological aspects of the evaluation and selection of MaOO algorithms are presented as the recommended solution on the basis of four distinct and successive phases. First, in the determination phase, the evaluation criteria of MaOO are collected, classified and grouped for testing experts’ consensus on the most suitable criteria. Second, the identification phase involves the process of establishing a decision matrix via a crossover of the ‘evaluation criteria’ and MaOO’, and the level of importance of each selective criteria and sub-criteria from phase one is computed to identify its weight value by using the best–worst method (BWM). Third, the development phase involves the creation of a decision matrix for MaOO selection on the basis of the integrated BWM and VIKOR method. Last, the validation phase involves the validation of the proposed solution.

Keywords: Evaluation; fuzzy Delphi; many-objective optimization; many-objective optimization problem; multi-criteria decision making (MCDM) (search for similar items in EconPapers)
Date: 2020
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0219622020300049
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:ijitdm:v:19:y:2020:i:06:n:s0219622020300049

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0219622020300049

Access Statistics for this article

International Journal of Information Technology & Decision Making (IJITDM) is currently edited by Yong Shi

More articles in International Journal of Information Technology & Decision Making (IJITDM) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().