Reinforcement learning for continuous-time mean-variance portfolio selection in a regime-switching market

Bo Wu and Lingfei Li

Journal of Economic Dynamics and Control, 2024, vol. 158, issue C

Abstract: We propose a reinforcement learning (RL) approach to solve the continuous-time mean-variance portfolio selection problem in a regime-switching market, where the market regime is unobservable. To encourage exploration for learning, we formulate an exploratory stochastic control problem with an entropy-regularized mean-variance objective. We obtain semi-analytical representations of the optimal value function and optimal policy, which involve unknown solutions to two linear parabolic partial differential equations (PDEs). We utilize these representations to parametrize the value function and policy for learning with the unknown solutions to the PDEs approximated based on polynomials. We develop an actor-critic RL algorithm to learn the optimal policy through interactions with the market environment. The algorithm carries out filtering to obtain the belief probability of the market regime and performs policy evaluation and policy gradient updates alternately. Empirical results demonstrate the advantages of our RL algorithm in relatively long-term investment problems over the classical control approach and an RL algorithm developed for the continuous-time mean-variance problem without considering regime switches.

Keywords: Reinforcement learning; Actor-critic; Mean-variance; Portfolio selection; Partial information; Regime-switching; Wonham's filter (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0165188923001938
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:dyncon:v:158:y:2024:i:c:s0165188923001938

DOI: 10.1016/j.jedc.2023.104787

Access Statistics for this article

Journal of Economic Dynamics and Control is currently edited by J. Bullard, C. Chiarella, H. Dawid, C. H. Hommes, P. Klein and C. Otrok

More articles in Journal of Economic Dynamics and Control from Elsevier
Bibliographic data for series maintained by Catherine Liu ().