Structural Properties of Stochastic Dynamic Programs

James E. Smith () and Kevin F. McCardle ()
Additional contact information
James E. Smith: Fuqua School of Business, Duke University, Box 90120, Durham, North Carolina 27708-0120
Kevin F. McCardle: The Anderson School at UCLA, 110 Westwood Plaza, Suite D520, Los Angeles, California 90095-1481

Operations Research, 2002, vol. 50, issue 5, 796-809

Abstract: In Markov models of sequential decision processes, one is often interested in showing that the value function is monotonic, convex, and/or supermodular in the state variables. These kinds of results can be used to develop a qualitative understanding of the model and characterize how the results will change with changes in model parameters. In this paper we present several fundamental results for establishing these kinds of properties. The results are, in essence, "metatheorems" showing that the value functions satisfy property P if the reward functions satisfy property P and the transition probabilities satisfy a stochastic version of this property. We focus our attention on closed convex cone properties, a large class of properties that includes monotonicity, convexity, and supermodularity, as well as combinations of these and many other properties of interest.

Keywords: Dynamic programming: properties of stochastic models; Decision analysis: properties of sequential models (search for similar items in EconPapers)
Date: 2002
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (25)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.50.5.796.365 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:50:y:2002:i:5:p:796-809

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().