Economic Model Predictive Control as a Solution to Markov Decision Processes

Dirk Reinhardt (), Akhil S. Anand, Shambhuraj Sawant and Sébastien Gros ()
Additional contact information
Dirk Reinhardt: Norwegian University of Science and Technology (NTNU)
Akhil S. Anand: Norwegian University of Science and Technology (NTNU)
Shambhuraj Sawant: Norwegian University of Science and Technology (NTNU)
Sébastien Gros: Norwegian University of Science and Technology (NTNU)

Chapter Chapter 7 in Model Predictive Control, 2025, pp 157-189 from Springer

Abstract: Abstract MDPs offer a fairly generic and powerful framework to discuss the notion of optimal policies for dynamic systems, in particular when the dynamics are stochastic. However, computing the optimal policy of an MDP can be very difficult due to the curse of dimensionality present in solving the underlying Bellman equations. MPC is a very popular technique for building control policies for complex dynamic systems. Historically, MPC has focused on constraint satisfaction and steering dynamic systems towards a user-defined reference. More recently, EMPC was proposed as a computationally tractable way of building optimal policies for dynamic systems. When stochasticity is present, EMPC is close to the MDP framework. In that context, EMPC can be construed as a tractable heuristic to provide approximate solutions to MDPs. However, there is arguably a knowledge gap in the literature regarding these approximate solutions and the conditions for an EMPC scheme to achieve closed-loop optimality. This chapter aims to clarify this approximation pedagogically, to provide the conditions for EMPC to deliver optimal policies, and to explore some of their consequences.

Date: 2025
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:dymchp:978-3-031-85256-5_7

Ordering information: This item can be ordered from
http://www.springer.com/9783031852565

DOI: 10.1007/978-3-031-85256-5_7

Access Statistics for this chapter

More chapters in Dynamic Modeling and Econometrics in Economics and Finance from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().