Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks

Oladayo S. Ajani, Sung-ho Hur () and Rammohan Mallipeddi ()
Additional contact information
Oladayo S. Ajani: School of Electronics Engineering, Kyungpook National University, Daegu 37224, Republic of Korea
Sung-ho Hur: School of Electronics Engineering, Kyungpook National University, Daegu 37224, Republic of Korea
Rammohan Mallipeddi: School of Electronics Engineering, Kyungpook National University, Daegu 37224, Republic of Korea

Mathematics, 2023, vol. 11, issue 23, 1-13

Abstract: Domain randomization in the context of Reinforcement learning (RL) involves training RL agents with randomized environmental properties or parameters to improve the generalization capabilities of the resulting agents. Although domain randomization has been favorably studied in the literature, it has been studied in terms of varying the operational characters of the associated systems or physical dynamics rather than their environmental characteristics. This is counter-intuitive as it is unrealistic to alter the mechanical dynamics of a system in operation. Furthermore, most works were based on cherry-picked environments within different classes of RL tasks. Therefore, in this work, we investigated domain randomization by varying only the properties or parameters of the environment rather than varying the mechanical dynamics of the featured systems. Furthermore, the analysis conducted was based on all six RL locomotion tasks. In terms of training the RL agents, we employed two proven RL algorithms (SAC and TD3) and evaluated the generalization capabilities of the resulting agents on several train–test scenarios that involve both in-distribution and out-distribution evaluations as well as scenarios applicable in the real world. The results demonstrate that, although domain randomization favors generalization, some tasks only require randomization from low-dimensional distributions while others require randomization from high-dimensional randomization. Hence the question of what level of randomization is optimal for any given task becomes very important.

Keywords: generalization; deep reinforcement learning; dynamic environments; locomotion; domain randomization (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/23/4744/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/23/4744/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:23:p:4744-:d:1286414

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().