Markov decision processes with quasi-hyperbolic discounting
Anna Jaśkiewicz () and
Andrzej S. Nowak ()
Additional contact information
Anna Jaśkiewicz: Wrocław University of Science and Technology
Andrzej S. Nowak: Faculty of Mathematics, Computer Science and Econometrics, University of Zielona Góra
Finance and Stochastics, 2021, vol. 25, issue 2, No 1, 189-229
Abstract We study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting.
Keywords: Markov decision process; Markov perfect equilibrium; Stochastic economic growth; 60J20; 91A10; 91A13; 91A25; 91B51; 91B62; 91G10; 91G80 (search for similar items in EconPapers)
JEL-codes: C61 C72 C73 G11 (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
http://link.springer.com/10.1007/s00780-020-00443-2 Abstract (text/html)
Access to the full text of the articles in this series is restricted.
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:spr:finsto:v:25:y:2021:i:2:d:10.1007_s00780-020-00443-2
Ordering information: This journal article can be ordered from
http://www.springer. ... ance/journal/780/PS2
Access Statistics for this article
Finance and Stochastics is currently edited by M. Schweizer
More articles in Finance and Stochastics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().