Undiscounted reinforcement learning for infinite-time optimal output tracking and disturbance rejection of discrete-time LTI systems with unknown dynamics

Ali Amirparast and S. Kamal Hosseini Sani

International Journal of Systems Science, 2023, vol. 54, issue 10, 2175-2195

Abstract: This paper proposes a novel control structure to solve the infinite-time linear quadratic tracking (LQT) problem. The major challenge in the LQT problem is the boundedness issue of the cost function in an infinite time framework. In many studies, a discount factor is utilised to overcome the challenge. However, it can affect the stability of the closed-loop system and the steady-state error. This paper proposes an optimal control structure that guarantees zero steady-state error with bounded cost function without utilising the discount factor. The optimal gains of the proposed control structure are computed via model-based and model-free reinforcement learning (RL) algorithms. As a novelty in model-based RL algorithms, a model predictive RL algorithm is proposed to reduce the number of iterations in the learning phase. A model-free reinforcement learning algorithm is utilised to obtain optimal control for tracking the reference online and without any knowledge of system dynamics. Finally, the simulation results verify the advantages of the proposed optimal control structure.

Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00207721.2023.2221240 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tsysxx:v:54:y:2023:i:10:p:2175-2195

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/TSYS20

DOI: 10.1080/00207721.2023.2221240

Access Statistics for this article

International Journal of Systems Science is currently edited by Visakan Kadirkamanathan

More articles in International Journal of Systems Science from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().