ACTION DISCOVERY FOR SINGLE AND MULTI-AGENT REINFORCEMENT LEARNING

Bikramjit Banerjee () and Landon Kraemer ()
Additional contact information
Bikramjit Banerjee: School of Computing, The University of Southern Mississippi, Hattiesburg, MS 39406-0001, USA
Landon Kraemer: School of Computing, The University of Southern Mississippi, Hattiesburg, MS 39406-0001, USA

Advances in Complex Systems (ACS), 2011, vol. 14, issue 02, pages 279-305

Abstract: The design of reinforcement learning solutions to many problems artificially constrain the action set available to an agent, in order to limit the exploration/sample complexity. While exploring, if an agent can discover new actions that can break through the constraints of its basic/atomic action set, then the quality of the learned decision policy could improve. On the flipside, considering all possible non-atomic actions might explode the exploration complexity. We present a novel heuristic solution to this dilemma, and empirically evaluate it in grid navigation tasks. In particular, we show that both the solution quality and the sample complexity improve significantly when basic reinforcement learning is coupled with action discovery. Our approach relies on reducing the number of decision points, which is particularly suited for multiagent coordination learning, since agents tend to learn more easily with fewer coordination problems (CPs). To demonstrate this we extend action discovery to multi-agent reinforcement learning. We show that Joint Action Learners (JALs) indeed learn coordination policies of higher quality with lower sample complexity when coupled with action discovery, in a multi-agent box-pushing task.

Keywords: Multi-agent reinforcement learning; multi-agent systems (search for similar items in EconPapers)
Date: 2011
References: Add references at CitEc
Citations Track citations by RSS feed

Downloads: (external link)
http://www.worldscinet.com/cgi-bin/details.cgi?type=pdf&id=pii:S0219525911002937 (application/pdf)
http://www.worldscinet.com/cgi-bin/details.cgi?typ ... ii:S0219525911002937 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: http://EconPapers.repec.org/RePEc:wsi:acsxxx:v:14:y:2011:i:02:p:279-305

Ordering information: This journal article can be ordered from

Access Statistics for this article

Advances in Complex Systems (ACS) is edited by Frank Schweitzer

More articles in Advances in Complex Systems (ACS) from World Scientific Publishing Co. Pte. Ltd.
Series data maintained by Tai Tone Lim ().