Computing the Expected Markov Reward Rates with Stationarity Detection and Relative Error Control

Víctor Suñé ()
Additional contact information
Víctor Suñé: Universitat Politècnica de Catalunya

Methodology and Computing in Applied Probability, 2017, vol. 19, issue 2, 445-485

Abstract: Abstract By combining in a novel way the randomization method with the stationary detection technique, we develop two new algorithms for the computation of the expected reward rates of finite, irreducible Markov reward models, with control of the relative error. The first algorithm computes the expected transient reward rate and the second one computes the expected averaged reward rate. The algorithms are numerically stable. Further, it is argued that, from the point of view of run-time computational cost, for medium-sized and large Markov reward models, we can expect the algorithms to be better than the only variant of the randomization method that allows to control the relative error and better than the approach that consists in employing iteratively the currently existing algorithms that use the randomization method with stationarity detection but allow to control the absolute error. The performance of the new algorithms is illustrated by means of examples, showing that the algorithms can be not only faster but also more efficient than the alternatives in terms of run-time computational cost in relation to accuracy.

Keywords: Markov reward model; Markov chain; Expected reward rate; Relative error; Randomization; Stationarity detection; 60J28; 60J22; 65C40 (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11009-016-9490-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:metcap:v:19:y:2017:i:2:d:10.1007_s11009-016-9490-y

Ordering information: This journal article can be ordered from
https://www.springer.com/journal/11009

DOI: 10.1007/s11009-016-9490-y

Access Statistics for this article

Methodology and Computing in Applied Probability is currently edited by Joseph Glaz

More articles in Methodology and Computing in Applied Probability from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().