Application of Monte Carlo stochastic optimization (MOST) to deep learning

Sin-ichi Inage and Hana Hebishima

Mathematics and Computers in Simulation (MATCOM), 2022, vol. 199, issue C, 257-271

Abstract: In this paper, we propose a new optimization method based on the Monte Carlo method. The proposed method is applied to several benchmark problems, and the result of applying it to the optimization of neural network is reported. Deep machine learning using neural networks is one of the important keywords to promote innovation in today’s advanced information society. Therefore, research on large-scale, high-speed, and high-precision algorithms has been actively conducted. The author has developed an optimization method in which the search region for multivariate parameters constituting the objective function is divided into two regions for each parameter, the integral values of each divided region are numerically calculated by the Monte Carlo method, the magnitude of each integral value is compared, and the optimum point is judged to be in a small region. The proposed method was applied to 50 variable benchmark functions (Schwefel and Ackley Functions), and was compared with the results of genetic algorithm (GA) which was a representative of existing optimization methods. As a result, it was confirmed that the proposed method is faster and more accurate than GA. In addition, the proposed method is applied to machine learning by neural networks, specifically XOR gate circuits and IRS classification problems, and verified. The neural network optimized by MOST reproduced teacher data and test data faster and more accurately than conventional Adam and genetic algorithms.

Keywords: Optimization algorithms; Neural network; Deep learning; Monte Carlo method; Genetic algorithm; Adam (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378475422001124
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:matcom:v:199:y:2022:i:c:p:257-271

DOI: 10.1016/j.matcom.2022.03.013

Access Statistics for this article

Mathematics and Computers in Simulation (MATCOM) is currently edited by Robert Beauwens

More articles in Mathematics and Computers in Simulation (MATCOM) from Elsevier
Bibliographic data for series maintained by Catherine Liu ().