Technical Note—Undiscounted Markov Renewal Programming Via Modified Successive Approximations

Thomas E. Morton
Additional contact information
Thomas E. Morton: Carnegie-Mellon University, Pittsburgh, Pennsylvania

Operations Research, 1971, vol. 19, issue 4, 1081-1089

Abstract: This note describes an efficient class of procedures for finding a solution to the functional equations \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland,xspace}\usepackage{amsmath,amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document}$$v^{*}_{i}=\max_{k} \biggl[q^{k}_{i}-g^{*}T^{k}_{i}+ \sum^{j=n}_{j=1}P^{k}_{ij}v_{j}^{*}\biggr],\quad 1\leq i \leq N,$$\end{document} of undiscounted Markov renewal programming. First, for the special case of a single possible policy, the problem is proved equivalent to solving two related ordinary Markov chain problems, which leads to an algorithm for the general problem whose exact form depends on the specification of a decision rule for alternation of two types of iterations. At one extreme, the technique is exactly “policy iteration,” with iterative techniques replacing solution of N equations for each improved policy; at the other extreme, the algorithm becomes essentially “value iteration,” generalizing the method of successive approximations proposed by D. J. White for Markovian decision processes. The latter version of the technique is related to another generalization being currently proposed by Paul J. Schweitzer; the methods being proposed here, however, do not deteriorate when the minimum transition time between states becomes very small.

Date: 1971
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.19.4.1081 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:19:y:1971:i:4:p:1081-1089

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().