Quadratic Tracking Control of Linear Stochastic Systems with Unknown Dynamics Using Average Off-Policy Q-Learning Method

Longyan Hao, Chaoli Wang () and Yibo Shi
Additional contact information
Longyan Hao: Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
Chaoli Wang: Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
Yibo Shi: Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Mathematics, 2024, vol. 12, issue 10, 1-21

Abstract: This article investigates the optimal tracking control problem for data-based stochastic discrete-time linear systems. An average off-policy Q-learning algorithm is proposed to solve the optimal control problem with random disturbances. Compared with the existing off-policy reinforcement learning (RL) algorithm, the proposed average off-policy Q-learning algorithm avoids the assumption of an initial stability control. First, a pole placement strategy is used to design an initial stable control for systems with unknown dynamics. Second, the initial stable control is used to design a data-based average off-policy Q-learning algorithm. Then, this algorithm is used to solve the stochastic linear quadratic tracking (LQT) problem, and a convergence proof of the algorithm is provided. Finally, numerical examples show that this algorithm outperforms other algorithms in a simulation.

Keywords: average off-policy Q-learning; linear quadratic tracking; data based control; reinforcement learning; stochastic linear system (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/10/1533/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/10/1533/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:10:p:1533-:d:1394587

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().