MULTIAGENT LEARNING FOR BLACK BOX SYSTEM REWARD FUNCTIONS

Kagan Tumer () and Adrian Agogino ()
Additional contact information
Kagan Tumer: Oregon State University, 204 Rogers Hall, Corvallis, Oregon 97331, USA
Adrian Agogino: UCSC, NASA Ames Research Center, Mailstop 269-3, Moffett Field, California 94035, USA

Advances in Complex Systems (ACS), 2009, vol. 12, issue 04n05, 475-492

Abstract: In large, distributed systems composed of adaptive and interactive components (agents), ensuring the coordination among the agents so that the system achieves certain performance objectives is a challenging proposition. The key difficulty to overcome in such systems is one of credit assignment: How to apportion credit (or blame) to a particular agent based on the performance of the entire system. In this paper, we show how this problem can be solved in general for a large class of reward functions whose analytical form may be unknown (hence "black box" reward). This method combines the salient features of global solutions (e.g. "team games") which are broadly applicable but provide poor solutions in large problems with those of local solutions (e.g. "difference rewards") which learn quickly, but can be computationally burdensome. We introduce two estimates for local rewards for a class of problems where the mapping from the agent actions to system reward functions can be decomposed into a linear combination of nonlinear functions of the agents' actions. We test our method's performance on a distributed marketing problem and an air traffic flow management problem and show a 44% performance improvement over team games and a speedup of ordernfor difference rewards (for annagent system).

Keywords: Multiagent learning; black box reward functions; multiagent coordination (search for similar items in EconPapers)
Date: 2009
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0219525909002295
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:acsxxx:v:12:y:2009:i:04n05:n:s0219525909002295

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0219525909002295

Access Statistics for this article

Advances in Complex Systems (ACS) is currently edited by Frank Schweitzer

More articles in Advances in Complex Systems (ACS) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().