MULTIAGENT LEARNING FOR BLACK BOX SYSTEM REWARD FUNCTIONS

Kagan Tumer () and Adrian Agogino ()
Additional contact information
Kagan Tumer: Oregon State University, 204 Rogers Hall, Corvallis, Oregon 97331, USA
Adrian Agogino: UCSC, NASA Ames Research Center, Mailstop 269-3, Moffett Field, California 94035, USA

Advances in Complex Systems (ACS), 2009, vol. 12, issue 04, pages 475-492

Abstract: In large, distributed systems composed of adaptive and interactive components (agents), ensuring the coordination among the agents so that the system achieves certain performance objectives is a challenging proposition. The key difficulty to overcome in such systems is one of credit assignment: How to apportion credit (or blame) to a particular agent based on the performance of the entire system. In this paper, we show how this problem can be solved in general for a large class of reward functions whose analytical form may be unknown (hence "black box" reward). This method combines the salient features of global solutions (e.g. "team games") which are broadly applicable but provide poor solutions in large problems with those of local solutions (e.g. "difference rewards") which learn quickly, but can be computationally burdensome. We introduce two estimates for local rewards for a class of problems where the mapping from the agent actions to system reward functions can be decomposed into a linear combination of nonlinear functions of the agents' actions. We test our method's performance on a distributed marketing problem and an air traffic flow management problem and show a 44% performance improvement over team games and a speedup of order n for difference rewards (for an n agent system).

Keywords: Multiagent learning; black box reward functions; multiagent coordination (search for similar items in EconPapers)
Date: 2009

Downloads: (external link)
http://www.worldscinet.com/cgi-bin/details.cgi?type=pdf&id=pii:S0219525909002295 (application/pdf)
http://www.worldscinet.com/cgi-bin/details.cgi?typ ... ii:S0219525909002295 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: http://EconPapers.repec.org/RePEc:wsi:acsxxx:v:12:y:2009:i:04:p:475-492

Ordering information: This journal article can be ordered from

Access Statistics for this article

Advances in Complex Systems (ACS) is edited by Frank Schweitzer

More articles in Advances in Complex Systems (ACS) from World Scientific Publishing Co. Pte. Ltd.
Series data maintained by Tai Tone Lim ().