Computing the discounted return in markov and semi‐markov chains

Evan L. Porteus

Naval Research Logistics Quarterly, 1981, vol. 28, issue 4, 567-577

Abstract: This paper addresses the problem of computing the expected discounted return in finite Markov and semi‐Markov chains. The objective is to reveal insights into two questions. First, which iterative methods hold the most promise? Second, when are interative methods preferred to Gaussian elimination? A set of twenty‐seven randomly generated problems is used to compare the performance of the methods considered. The observations that apply to the problems generated here are as follows: Gauss‐Seidel is not preferred to Pre‐Jacobi in general. However, if the matrix is reordered in a certain way and the author's row sum extrapolation is used, then Gauss‐Seidel is preferred. Transforming a semi‐Markov problem into a Markov one using a transformation that comes from Schweitzer does not yield improved performance. A method analogous to symmetric successive overrelaxation (SSOR) in numerical analysis yields improved performance, especially when the row‐sum extrapolation is used only sparingly. This method is then compared to Gaussian elimination and is found to be superior for most of the problems generated.

Date: 1981
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/nav.3800280406

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:navlog:v:28:y:1981:i:4:p:567-577

Access Statistics for this article

More articles in Naval Research Logistics Quarterly from John Wiley & Sons
Bibliographic data for series maintained by Wiley Content Delivery ().