Markov Decision Processes and Stochastic Control Problems on Networks

Dmitrii Lozovanu and Stefan Wolfgang Pickl
Additional contact information
Dmitrii Lozovanu: Moldowa Academy of Science
Stefan Wolfgang Pickl: Universität der Bundeswehr München

Chapter Chapter 2 in Markov Decision Processes and Stochastic Positional Games, 2024, pp 125-244 from Springer

Abstract: Abstract In this chapter, we study a class of problems for Markov decision process models with finite state and action spaces. We consider finite and infinite horizon models. For a finite horizon model, the problem with an expected total reward optimization criterion is considered, which can be efficiently solved by using the backward dynamic programming technique. For infinite horizon models, two basic problems are studied: the problem with an expected total discounted reward optimization criterion and the problem with an expected average reward optimization criterion. We present some classical results concerned with determining the optimal solutions to these problems and show how these results can be extended for a class of control problems on networks. The main attention is addressed to the linear programming approach for Markov decision processes and control problems on networks. Our emphasis is on formulating and studying the infinite horizon decision problems in terms of stationary strategies. We show that infinite horizon Markov decision problems with average and discounted optimization criteria can be formulated in terms of stationary strategies as classical mathematical programming problems with quasi-monotonic (quasi-convex and quasi-concave) object functions and linear constraints. In the following, we show that such quasi-monotonic programming models for infinite horizon decision problems are useful for studying stochastic games with average and discounted payoffs.

Keywords: Markov decision processes; Control problems on networks; Iterative algorithms; Linear programming approach; Quasi-linear programming approach (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:isochp:978-3-031-40180-0_2

Ordering information: This item can be ordered from
http://www.springer.com/9783031401800

DOI: 10.1007/978-3-031-40180-0_2

Access Statistics for this chapter

More chapters in International Series in Operations Research & Management Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().