Markov Decision Processes

Vivek S. Borkar (), Vladimir Ejov (), Jerzy A. Filar () and Giang T. Nguyen ()
Additional contact information
Vivek S. Borkar: IIT, Powai
Vladimir Ejov: Flinders University
Jerzy A. Filar: Flinders University
Giang T. Nguyen: Université libre de Bruxelles

Chapter Chapter 4 in Hamiltonian Cycle Problem and Markov Chains, 2012, pp 49-66 from Springer

Abstract: Abstract Markov chains are useful in describing many discrete event stochastic processes; however, they are not exible enough to model situations where we have to make decisions to control the future trajectories of the system. For this reason, the theory of Markov decision processes (MDPs), also known as controlled Markov chains, has been developed. In particular, in the context of this book, we observe that in any given graph Hamiltonian cycles (if any) correspond to a family of spanning subgraphs inducing very special Markov chains whose probability transition matrices are a subset of permutation matrices possessing only a single ergodic class.

Keywords: Extreme Point; Markov Decision Process; Hamiltonian Cycle; Short Cycle; Expected Reward (search for similar items in EconPapers)
Date: 2012
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:isochp:978-1-4614-3232-6_4

Ordering information: This item can be ordered from
http://www.springer.com/9781461432326

DOI: 10.1007/978-1-4614-3232-6_4

Access Statistics for this chapter

More chapters in International Series in Operations Research & Management Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().