Technical Note—Successive Approximations in Value Determination for a Markov Decision Process

Theodore J. Sheskin
Additional contact information
Theodore J. Sheskin: Cleveland State University, Cleveland, Ohio

Operations Research, 1987, vol. 35, issue 5, 784-786

Abstract: We present a new procedure for solving the value-determination equations for an undiscounted Markov decision process by the successive approximations of a Gauss-Seidel or Jacobi iteration method. The key step is to use the Markov chain partitioning algorithm to compute the steady-state probabilities corresponding to a given policy. The gain, computed in terms of the steady-state probabilities, is substituted into the value-determination equations. After deleting one relative value and one value-determination equation, we show that the remaining system of equations satisfies the sufficient conditions for the convergence of a Gauss-Seidel or Jacobi iteration.

Keywords: 119; successive; approximations; in; value; determination (search for similar items in EconPapers)
Date: 1987
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.35.5.784 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:35:y:1987:i:5:p:784-786

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().