Procedures for the Solution of a Finite-Horizon, Partially Observed, Semi-Markov Optimization Problem

Chelsea C. White
Additional contact information
Chelsea C. White: University of Virginia, Charlottesville, Virginia

Operations Research, 1976, vol. 24, issue 2, 348-358

Abstract: This paper presents a finite-time-horizon, partially observed, stochastic optimization problem, where the core process is a controlled, finite-state, discrete-time semi-Markov process. We assume that times of control reset and noise corrupted observations of the core process occur at times of core process transition. The control employed at each time of core process transition is allowed to be functionally dependent on the sample path of the core process only through the history of corrupted observations. Conditions for optimality are stated. We show that the minimum expected cost accrued from a given time t until the terminal time is concave with respect to a sufficient statistic and satisfies a computationally useful functional form. Furthermore, if the set of values that the control vector may assume at each time of control reset is finite, then the expected cost is also piecewise linear with respect to the sufficient statistic. A numerical procedure for the finite-horizon, partially observed, Markov optimization problem is generalized to include the semi-Markov formulation. We give a result that justifies a well known technique for the determination of the optimal cost and an optimal control policy and discuss areas of application.

Date: 1976
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.24.2.348 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:24:y:1976:i:2:p:348-358

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().