Optimal Control of Partially Observable Semi-Markovian Failing Systems: An Analysis Using a Phase Methodology

Akram Khaleghei () and Michael Jong Kim ()
Additional contact information
Akram Khaleghei: Rotman School of Business, University of Toronto, Toronto, Ontario M5S 3E6, Canada
Michael Jong Kim: Sauder School of Business, University of British Columbia, Vancouver, British Columbia V6T 1Z2, Canada

Operations Research, 2021, vol. 69, issue 4, 1282-1304

Abstract: We formulate a maintenance control model as an optimal stopping problem under partial observations. The key challenge in our formulation is that the underlying state process is not restricted to be Markovian but rather is allowed to follow a semi-Markov process, which is more realistic in practice. Consequently, the stopping problem is not representable as a partially observable Markov decision process (POMDP) with finite state space, a commonly adopted modeling framework in the maintenance optimization literature; it constitutes a partially observable semi-Markov decision process (POSMDP), a problem class that in general is both computationally intractable and not amenable to structural analysis. In this paper, we develop a new analysis approach based on a phase methodology where the idea is to view the intractable POSMDP as the limiting problem of a sequence of tractable POMDPs. We show how this approach allows us to (i) characterize the structural form of the optimal policy and (ii) efficiently compute the optimal policy and associated optimal value via successive approximation. In particular, we show that the optimal control policy can be represented as a control limit policy that monitors the estimated conditional reliability at each decision epoch, and, by exploiting this structure, we develop an efficient computational approach to solve for the optimal control limit and corresponding optimal value. Numerical comparisons are provided that show substantial improvement over existing policies.

Keywords: dynamic programming/optimal control; Stochastic Models; condition-based maintenance; partially observable semi-Markov decision process; likelihood ratio ordering; Bayesian control chart; conditional reliability function (search for similar items in EconPapers)
Date: 2021
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2020.2086 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:69:y:2021:i:4:p:1282-1304

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().