 Vector-based navigation using grid-like representations in artificial agentsAndrea Banino (),
Caswell Barry (),
Benigno Uria,
Charles Blundell,
Timothy Lillicrap,
Piotr Mirowski,
Alexander Pritzel,
Martin J. Chadwick,
Thomas Degris,
Joseph Modayil,
Greg Wayne,
Hubert Soyer,
Fabio Viola,
Brian Zhang,
Ross Goroshin,
Neil Rabinowitz,
Razvan Pascanu,
Charlie Beattie,
Stig Petersen,
Amir Sadik,
Stephen Gaffney,
Helen King,
Koray Kavukcuoglu,
Demis Hassabis,
Raia Hadsell and
Dharshan Kumaran ()
Nature, 2018, vol. 557, issue 7705, 429-433 Abstract: Abstract Deep neural networks have achieved impressive successes in fields ranging from object recognition to complex games such as Go1,2. Navigation, however, remains a substantial challenge for artificial agents, with deep neural networks trained by reinforcement learning3–5 failing to rival the proficiency of mammalian spatial behaviour, which is underpinned by grid cells in the entorhinal cortex6. Grid cells are thought to provide a multi-scale periodic representation that functions as a metric for coding space7,8 and is critical for integrating self-motion (path integration)6,7,9 and planning direct trajectories to goals (vector-based navigation)7,10,11. Here we set out to leverage the computational functions of grid cells to develop a deep reinforcement learning agent with mammal-like navigational abilities. We first trained a recurrent network to perform path integration, leading to the emergence of representations resembling grid cells, as well as other entorhinal cell types12. We then showed that this representation provided an effective basis for an agent to locate goals in challenging, unfamiliar, and changeable environments—optimizing the primary objective of navigation through deep reinforcement learning. The performance of agents endowed with grid-like representations surpassed that of an expert human and comparison agents, with the metric quantities necessary for vector-based navigation derived from grid-like units within the network. Furthermore, grid-like representations enabled agents to conduct shortcut behaviours reminiscent of those performed by mammals. Our findings show that emergent grid-like representations furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for vector-based navigation7,10,11, demonstrating that the latter can be combined with path-based strategies to support navigation in challenging environments. Date: 2018 Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:nat:nature:v:557:y:2018:i:7705:d:10.1038_s41586-018-0102-6 Ordering information: This journal article can be ordered from DOI: 10.1038/s41586-018-0102-6 Access Statistics for this article Nature is currently edited by Magdalena Skipper More articles in Nature from Nature |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.