Continuous-time reinforcement learning for robust control under worst-case uncertainty

Adolfo Perrusquía and Wen Yu

International Journal of Systems Science, 2021, vol. 52, issue 4, 770-784

Abstract: Reinforcement learning (RL) is an effective method to design a robust controller for unknown nonlinear systems. Uncertainty in the worst case requires a large state-action space. Hence, it is natural to use continuous-time RL methods rather than the discretisation of the spaces. In this paper, we propose a novel continuous-time RL using neural network approximation. Our method uses worst-case uncertainty to train the continuous-time RL algorithm. The backward Euler approximation is used to approximate the time derivative of the value function. Compared with the actor–critic (AC) algorithm, our method finds the robust control policy in the presence of worst-case uncertainty by taking into account the applied actions. It is shown that the AC algorithm finds the robust controller in less episodes, but its robustness is less than the results presented by our approach. The convergence of the proposed algorithm is analysed using the contraction property and differential equation techniques. The experiments show that our approach is more robust than the model-based LQR method and the well-known AC method.

Date: 2021
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00207721.2020.1839142 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tsysxx:v:52:y:2021:i:4:p:770-784

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/TSYS20

DOI: 10.1080/00207721.2020.1839142

Access Statistics for this article

International Journal of Systems Science is currently edited by Visakan Kadirkamanathan

More articles in International Journal of Systems Science from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().