A Continuous-Time Markov Decision Process for Infrastructure Surveillance

Jonathan Ott ()
Additional contact information
Jonathan Ott: Karlsruhe Institute of Technology

A chapter in Operations Research Proceedings 2010, 2011, pp 327-332 from Springer

Abstract: Abstract We consider the dynamics of threat of an infrastructure consisting of various sectors. The dynamics are modelled by a continuous-time Markov decision process (CMDP). A key property of the model is that dependencies between sectors are taken into account. The actions are determined by the number of the available security staff. The decision maker has to allocate the security staff to the sectors which leads to a resource allocation problem. The CMDP criterion is to minimize expected discounted total cost over an infinite horizon. This problem is computable only for very small infrastructures. To solve the problem for larger infrastructures, an index for a specific restless bandit model is introduced. The index can be easily implemented. The index is used as a basis of a heuristics which defines a sub-optimal decision rule for the surveillance task. By using this heuristics, memory requirements are reduced in order to store the suboptimal decision rule efficiently.

Keywords: Decision Rule; Resource Allocation Problem; Threat Event; Threat Level; Approximate Dynamic Program (search for similar items in EconPapers)
Date: 2011
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:oprchp:978-3-642-20009-0_52

Ordering information: This item can be ordered from
http://www.springer.com/9783642200090

DOI: 10.1007/978-3-642-20009-0_52

Access Statistics for this chapter

More chapters in Operations Research Proceedings from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().