RHOASo: An Early Stop Hyper-Parameter Optimization Algorithm

Ángel Luis Muñoz Castañeda, Noemí DeCastro-García and David Escudero García
Additional contact information
Ángel Luis Muñoz Castañeda: Department of Mathematics, Universidad de León, 24007 León, Spain
Noemí DeCastro-García: Department of Mathematics, Universidad de León, 24007 León, Spain
David Escudero García: Research Institute of Applied Sciences in Cybersecurity (RIASC), Universidad de León, 24007 León, Spain

Mathematics, 2021, vol. 9, issue 18, 1-52

Abstract: This work proposes a new algorithm for optimizing hyper-parameters of a machine learning algorithm, RHOASo, based on conditional optimization of concave asymptotic functions. A comparative analysis of the algorithm is presented, giving particular emphasis to two important properties: the capability of the algorithm to work efficiently with a small part of a dataset and to finish the tuning process automatically, that is, without making explicit, by the user, the number of iterations that the algorithm must perform. Statistical analyses over 16 public benchmark datasets comparing the performance of seven hyper-parameter optimization algorithms with RHOASo were carried out. The efficiency of RHOASo presents the positive statistically significant differences concerning the other hyper-parameter optimization algorithms considered in the experiments. Furthermore, it is shown that, on average, the algorithm needs around 70 % of the iterations needed by other algorithms to achieve competitive performance. The results show that the algorithm presents significant stability regarding the size of the used dataset partition.

Keywords: hyperparameters; machine learning; optimization; inference (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/9/18/2334/pdf (application/pdf)
https://www.mdpi.com/2227-7390/9/18/2334/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:9:y:2021:i:18:p:2334-:d:639562

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().