A SEQUENTIAL DECISION PROBLEM BASED ON THE RATE DEPENDING ON A MARKOV PROCESS

Tōru Nakai
Additional contact information
Tōru Nakai: Department of Math & Informatics, Chiba University, Yayoi-cho, Inage-ku, Chiba 263-8522, Japan

Chapter 2 in Recent Advances in Stochastic Operations Research II, 2009, pp 11-30 from World Scientific Publishing Co. Pte. Ltd.

Abstract: AbstractIt is usual to grasp an activity of the public sector as a cycle of inputs, outputs and outcomes. The inputs are the resources, the outputs are the products achieved, and the outcome is the criterion to measure the results, but it is difficult to evaluate the outcome. A sequential expenditure problem on a Markov process will be considered, and a state of this process is closely related to the outcomes. This state can be changed by expending an additional amount, and it also changes according to a Markovian transition rule based on TP2. This stochastic order plays an important role in the Bayesian learning procedure for a partially observable Markov process. The dynamic programming formulation implies a recursive equation about the expected value obtainable under the optimal policy. There are some monotonic properties concerning this value and the optimal policy. Finally, we treat this problem on a partially observable Markov process with Bayesian learning after observing some properties under assumptions since the state can be changed by decisions. It is also possible to consider a monotonic property for this case.

Keywords: Operations Research; Uncertainty; Applied Probability; Stochastic Process; Optimization; Decision Science (search for similar items in EconPapers)
Date: 2009
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.worldscientific.com/doi/pdf/10.1142/9789812791672_0002 (application/pdf)
https://www.worldscientific.com/doi/abs/10.1142/9789812791672_0002 (text/html)
Ebook Access is available upon purchase.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:wschap:9789812791672_0002

Ordering information: This item can be ordered from

Access Statistics for this chapter

More chapters in World Scientific Book Chapters from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().