Bias and Variance Approximation in Value Function Estimates

Shie Mannor (), Duncan Simester (), Peng Sun () and John N. Tsitsiklis ()
Additional contact information
Shie Mannor: Department of Electrical and Computer Engineering, McGill University, Montreal, Quebec, Canada H3A 2A7
Duncan Simester: Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Peng Sun: Fuqua School of Business, Duke University, Durham, North Carolina 27708
John N. Tsitsiklis: Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139

Management Science, 2007, vol. 53, issue 2, 308-322

Abstract: We consider a finite-state, finite-action, infinite-horizon, discounted reward Markov decision process and study the bias and variance in the value function estimates that result from empirical estimates of the model parameters. We provide closed-form approximations for the bias and variance, which can then be used to derive confidence intervals around the value function estimates. We illustrate and validate our findings using a large database describing the transaction and mailing histories for customers of a mail-order catalog firm.

Keywords: value function; confidence interval; variance; bias (search for similar items in EconPapers)
Date: 2007
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (19)

Downloads: (external link)
http://dx.doi.org/10.1287/mnsc.1060.0614 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:53:y:2007:i:2:p:308-322

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().