Approximate Dynamic Programming by Practical Examples

Martijn R. K. Mes () and Arturo Pérez Rivera
Additional contact information
Martijn R. K. Mes: University of Twente
Arturo Pérez Rivera: University of Twente

Chapter Chapter 3 in Markov Decision Processes in Practice, 2017, pp 63-101 from Springer

Abstract: Abstract Computing the exact solution of an MDP model is generally difficult and possibly intractable for realistically sized problem instances. A powerful technique to solve the large scale discrete time multistage stochastic control processes is Approximate Dynamic Programming (ADP). Although ADP is used as an umbrella term for a broad spectrum of methods to approximate the optimal solution of MDPs, the common denominator is typically to combine optimization with simulation, use approximations of the optimal values of the Bellman’s equations, and use approximate policies. This chapter aims to present and illustrate the basics of these steps by a number of practical and instructive examples. We use three examples (1) to explain the basics of ADP, relying on value iteration with an approximation of the value functions, (2) to provide insight into implementation issues, and (3) to provide test cases for the reader to validate its own ADP implementations.

Keywords: Dynamic programming; Approximate dynamic programming; Stochastic optimization; Monte Carlo simulation; Curse of dimensionality (search for similar items in EconPapers)
Date: 2017
References: Add references at CitEc
Citations: View citations in EconPapers (6)

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:isochp:978-3-319-47766-4_3

Ordering information: This item can be ordered from
http://www.springer.com/9783319477664

DOI: 10.1007/978-3-319-47766-4_3

Access Statistics for this chapter

More chapters in International Series in Operations Research & Management Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().