Solving Finite-Horizon Discounted Non-Stationary MDPS

Bouchra El Akraoui () and Daoui Cherki ()
Additional contact information
Bouchra El Akraoui: Slimane University, Laboratory of Information Processing and Decision Support, Sultan Moulay, Morocco.
Daoui Cherki: Slimane University, Laboratory of Information Processing and Decision Support, Sultan Moulay, Morocco.

Folia Oeconomica Stetinensia, 2023, vol. 23, issue 1, 1-15

Abstract: Research background Markov Decision Processes (MDPs) are a powerful framework for modeling many real-world problems with finite-horizons that maximize the reward given a sequence of actions. Although many problems such as investment and financial market problems where the value of a reward decreases exponentially with time, require the introduction of interest rates. Purpose This study investigates non-stationary finite-horizon MDPs with a discount factor to account for fluctuations in rewards over time. Research methodology To consider the fluctuations of rewards with time, the authors define new nonstationary finite-horizon MDPs with a discount factor. First, the existence of an optimal policy for the proposed finite-horizon discounted MDPs is proven. Next, a new Discounted Backward Induction (DBI) algorithm is presented to find it. To enhance the value of their proposal, a financial model is used as an example of a finite-horizon discounted MDP and an adaptive DBI algorithm is used to solve it. Results The proposed method calculates the optimal values of the investment to maximize its expected total return with consideration of the time value of money. Novelty No existing studies have before examined dynamic finite-horizon problems that account for temporal fluctuations in rewards.

Keywords: Markov Decision Process; Dynamic Programming; Backward Induction algorithm (search for similar items in EconPapers)
JEL-codes: C02 C44 G11 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.2478/foli-2023-0001 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:vrs:foeste:v:23:y:2023:i:1:p:1-15:n:12

DOI: 10.2478/foli-2023-0001

Access Statistics for this article

Folia Oeconomica Stetinensia is currently edited by Waldemar Tarczyński

More articles in Folia Oeconomica Stetinensia from Sciendo
Bibliographic data for series maintained by Peter Golla ().