On reachability of Markov chains: A long-run average approach

M. Junca and D. Ávila
Additional contact information
D. Ávila: Université catholique de Louvain, LIDAM/CORE, Belgium

No 3179, LIDAM Reprints CORE from Université catholique de Louvain, Center for Operations Research and Econometrics (CORE)

Abstract: We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we characterize the domain of attraction and escape set of the system, and a generalization called p-domain of attraction, using the aforementioned value function. Next, we solve the problem of maximizing the probability of reaching a set $A$ while avoiding a set B. Finally, we consider a constrained version of the previous problem where we ask for the probability of reaching the set $B$ to be bounded. In the finite case, we use linear programming formulations to solve these problem.

Keywords: Electrical and Electronic Engineering; Computer Science Applications; Control and Systems Engineering (search for similar items in EconPapers)
Pages: 8
Date: 2021-08-24
Note: In: IEEE Transactions on Automatic Control, 2021
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cor:louvrp:3179

DOI: 10.1109/tac.2021.3071334

Access Statistics for this paper

More papers in LIDAM Reprints CORE from Université catholique de Louvain, Center for Operations Research and Econometrics (CORE) Voie du Roman Pays 34, 1348 Louvain-la-Neuve (Belgium). Contact information at EDIRC.
Bibliographic data for series maintained by Alain GILLIS ().