A Value Iteration Algorithm for Stochastic Linear Quadratic Regulator

Hongxia Wang (), Yihang Liu () and Xiangqian Liu ()
Additional contact information
Hongxia Wang: Shandong University of Science and Technology
Yihang Liu: CAS Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences
Xiangqian Liu: Shandong University of Science and Technology

Journal of Optimization Theory and Applications, 2025, vol. 207, issue 2, No 1, 14 pages

Abstract: Abstract In this paper, we propose a novel value iteration algorithm for online adaptive optimal control of discrete-time stochastic linear quadratic regulator (LQR) problems. The algorithm iteratively solves the algebraic Riccati equation (ARE) using online information of states and inputs, without requiring the knowledge of the system dynamics. It does not require a discount factor because the issue of excitation noise bias is not present in our algorithm. Firstly, we review the optimal solution for the stochastic LQR problem. Secondly, we offer an offline model-based algorithm for solving ARE and prove its convergence. Thirdly, we present a data-driven online value iteration algorithm for solving ARE. Finally, we evaluate the proposed algorithm by an example.

Keywords: Stochastic linear quadratic regulation; Reinforcement learning; Value iteration; Policy iteration (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10957-025-02777-3 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:207:y:2025:i:2:d:10.1007_s10957-025-02777-3

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-025-02777-3

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().