# Convergence Properties of Policy Iteration

*Manuel Santos* () and
*John Rust* ()

Additional contact information

Manuel Santos: W. P. Carey School of Business Department of Economics, http://wpcarey.asu.edu/Directory/stafffaculty.cfm?cobid=1039501

Working Papers from Department of Economics, W. P. Carey School of Business, Arizona State University

**Abstract:**
This paper analyzes the asymptotic convergence properties of policy iteration in a class of stationary, infinite-horizon Markovian decision problems that arise in optimal growth theory. These problems have continuous state and control variables, and must therefore be discretized in order to compute an approximate solution. The discretization converts a potentially infinite dimensional fixed-point problem to a finite dimensional problem defined on a finite grid of points in the state space, and it may thus render inapplicable known convergence results for policy iteration such as those of Puterman and Brumelle (1979). Under certain regularity conditions, we prove that for piecewise linear interpolation, policy iteration converges quadratically, i.e. the sequence of errors en = |Vn - V*| (where Vn is an approximate value function produced from the nth policy iteration step) satisfies en+1 = Le2n for all n. We show how the constant L depends on the grid size of the discretization. Also, under more general conditions we establish that convergence is superlinear. We illustrate the theoretical results with numerical experiments that compare the performance of policy iteration and the method of successive approximations. The quantitative results are consistent with theoretical predictions.

**New Economics Papers:** this item is included in nep-cmp

**References:** View references in EconPapers View complete reference list from CitEc

**Citations** Track citations by RSS feed

**Downloads:** (external link)

http://wpcarey.asu.edu/tools/mytools/pubs_admin/FILES/manuel-rust.pdf

**Our link check indicates that this URL is bad, the error code is: 404 Not Found (http://wpcarey.asu.edu/tools/mytools/pubs_admin/FILES/manuel-rust.pdf [301 Moved Permanently]--> https://wpcarey.asu.edu/tools/mytools/pubs_admin/FILES/manuel-rust.pdf)**

**Related works:**

This item may be available elsewhere in EconPapers: Search for items with the same title.

**Export reference:** BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text

**Persistent link:** http://EconPapers.repec.org/RePEc:asu:wpaper:2133377

Access Statistics for this paper

More papers in Working Papers from Department of Economics, W. P. Carey School of Business, Arizona State University Contact information at EDIRC.

Series data maintained by Steve Salik ().