Joint chance-constrained Markov decision processes

V Varagapriya (), Vikas Vikram Singh () and Abdel Lisser ()
Additional contact information
V Varagapriya: Indian Institute of Technology Delhi
Vikas Vikram Singh: Indian Institute of Technology Delhi
Abdel Lisser: Universite Paris Saclay

Annals of Operations Research, 2023, vol. 322, issue 2, No 17, 1013-1035

Abstract: Abstract We consider a finite state-action uncertain constrained Markov decision process under discounted and average cost criteria. The running costs are defined by random variables and the transition probabilities are known. The uncertainties present in the objective function and the constraints are modelled using chance constraints. We assume that the random cost vectors follow multivariate elliptically symmetric distributions and dependence among the random constraints is driven by a Gumbel–Hougaard copula. We propose two second order cone programming problems whose optimal values give lower and upper bounds of the optimal value of the uncertain constrained Markov decision process. As an application, we study a stochastic version of a service and admission control problem in a queueing system. The proposed approximation methods are illustrated on randomly generated instances of queueing control problem as well as on well known class of Markov decision problems known as Garnets.

Keywords: Constrained Markov decision process; Joint chance constraint; Second order cone programming problem; Copula; Elliptical distribution; Queueing problem (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10479-022-05025-3 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:annopr:v:322:y:2023:i:2:d:10.1007_s10479-022-05025-3

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10479

DOI: 10.1007/s10479-022-05025-3

Access Statistics for this article

Annals of Operations Research is currently edited by Endre Boros

More articles in Annals of Operations Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().