Exploratory Optimal Stopping: A Singular Control Formulation

Jodi Dianetti, Giorgio Ferrari and Renyuan Xu
Additional contact information
Jodi Dianetti: Center for Mathematical Economics, Bielefeld University
Giorgio Ferrari: Center for Mathematical Economics, Bielefeld University
Renyuan Xu: Center for Mathematical Economics, Bielefeld University

No 740, Center for Mathematical Economics Working Papers from Center for Mathematical Economics, Bielefeld University

Abstract: This paper explores continuous-time and state-space optimal stopping problems from a reinforcement learning perspective. We begin by formulating the stopping problem using randomized stopping times, where the decision maker’s control is represented by the probability of stopping within a given time—specifically, a bounded, non-decreasing, càdlàg control process. To encourage exploration and facilitate learning, we introduce a regularized version of the problem by penalizing the performance criterion with the cumulative residual entropy of the randomized stopping time. The regularized problem takes the form of an (n + 1)-dimensional degenerate singular stochastic control with finite-fuel. We address this through the dynamic programming principle, which enables us to identify the unique optimal exploratory strategy. For the specific case of a real option problem, we derive a semi-explicit solution to the regularized problem, allowing us to assess the impact of entropy regularization and analyze the vanishing entropy limit. Finally, we propose a reinforcement learning algorithm based on policy iteration. We show both policy improvement and policy convergence results for our proposed algorithm.

Keywords: Optimal stopping; singular stochastic control; free boundary problem; entropy regularization; reinforcement learning (search for similar items in EconPapers)
Pages: 51
Date: 2025-08-15
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://pub.uni-bielefeld.de/download/3006178/3006179 First Version, 2024 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bie:wpaper:740

Access Statistics for this paper

More papers in Center for Mathematical Economics Working Papers from Center for Mathematical Economics, Bielefeld University Contact information at EDIRC.
Bibliographic data for series maintained by Bettina Weingarten ().