Augmented truncation approximations to the solution of Poisson’s equation for Markov chains

Jinpeng Liu, Yuanyuan Liu and Yiqiang Q. Zhao

Applied Mathematics and Computation, 2022, vol. 414, issue C

Abstract: Poisson’s equation has a lot of applications in various areas, such as Markov decision theory, perturbation theory, central limit theorems (CLTs), etc. Usually it is hard to derive the explicit expression of the solution of Poisson’s equation for a Markov chain on an infinitely many state space. Here we will present a computational framework for the solution for both discrete-time Markov chains (DTMCs) and continuous-time Markov chains (CTMCs), by developing the technique of augmented truncation approximations. The censored Markov chain and the linear augmentation to some columns are shown to be effective truncation approximation schemes. Moreover, the convergence to the variance constant in CLTs are also considered. Finally the results obtained are applied to discrete-time single-birth processes and continuous-time single-death processes.

Keywords: Markov chains; Truncation approximation; Poisson’s equation; Central limit theorem; Single-birth processes; Single-death processes (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0096300321006949
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:apmaco:v:414:y:2022:i:c:s0096300321006949

DOI: 10.1016/j.amc.2021.126610

Access Statistics for this article

Applied Mathematics and Computation is currently edited by Theodore Simos

More articles in Applied Mathematics and Computation from Elsevier
Bibliographic data for series maintained by Catherine Liu ().