AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES

Jan Kaluski
Additional contact information
Jan Kaluski: The Silesian Technical University

No 235, Computing in Economics and Finance 2000 from Society for Computational Economics

Abstract: In the paper, a theorem about the existence of the ergodic and difference matrices of the finite state discounted Markov decision processes had been formulated and proved. On the basis of this theorem an analytical method to calculate these matrices is presented. The theorem allows the distribution of the overall value into two parts: the so-called "constant" part, which represents a part of the value related to the ergodic matrix and the "variable" part which represents a sum of the difference matrices. On the basis of the mentioned analytical method a new performance index to the discounted optimal control Markov problem is proposed and some interpretation of the received results is given. The proposed new performance index is formulated as a quotient of the distinguished parts of the overall value. The method is illustrated by two simple examples.

Date: 2000-07-05
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://fmwww.bc.edu/cef00/papers/paper235.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sce:scecf0:235

Access Statistics for this paper

More papers in Computing in Economics and Finance 2000 from Society for Computational Economics CEF 2000, Departament d'Economia i Empresa, Universitat Pompeu Fabra, Ramon Trias Fargas, 25,27, 08005, Barcelona, Spain. Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F. Baum ().