A Model for Multi-timescaled Sequential Decision-making Processes with Adversary

H.S. Chang

Mathematical and Computer Modelling of Dynamical Systems, 2004, vol. 10, issue 3-4, 287-302

Abstract: Extending the multi-timescale model proposed by the author et al. in the context of Markov decision processes, this paper proposes a simple analytical model called M timescale two-person zero-sum Markov Games (MMGs) for hierarchically structured sequential decision-making processes in two players' competitive situations where one player (the minimizer) wishes to minimize their cost that will be paid to the adversary (the maximizer). In this hierarchical model, for each player, decisions in each level in the M -level hierarchy are made in M different discrete timescales and the state space and the control space of each level in the hierarchy are non-overlapping with those of the other levels, respectively, and the hierarchy is structured in a "pyramid" sense such that a decision made at level m (slower timescale) state and/or the state will affect the evolutionary decision making process of the lower-level m +1 (faster timescale) until a new decision is made at the higher level but the lower-level decisions themselves do not affect the transition dynamics of higher levels. The performance produced by the lower-level decisions will affect the higher level decisions for each player. A hierarchical objective function for the minimizer and the maximizer is defined, and from this we define "multi-level equilibrium value function" and derive a "multi-level equilibrium equation". We also discuss how to solve hierarchical games exactly.

Date: 2004
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/13873950412331335261 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:nmcmxx:v:10:y:2004:i:3-4:p:287-302

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/NMCM20

DOI: 10.1080/13873950412331335261

Access Statistics for this article

Mathematical and Computer Modelling of Dynamical Systems is currently edited by I. Troch

More articles in Mathematical and Computer Modelling of Dynamical Systems from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().