Random synaptic feedback weights support error backpropagation for deep learning

Timothy P. Lillicrap (), Daniel Cownden, Douglas B. Tweed and Colin J. Akerman ()
Additional contact information
Timothy P. Lillicrap: University of Oxford
Daniel Cownden: School of Biology, University of St Andrews, Harold Mitchel Building, St Andrews
Douglas B. Tweed: University of Toronto
Colin J. Akerman: University of Oxford

Nature Communications, 2016, vol. 7, issue 1, 1-10

Abstract: Abstract The brain processes information through multiple layers of neurons. This deep architecture is representationally powerful, but complicates learning because it is difficult to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame by multiplying error signals with all the synaptic weights on each neuron’s axon and further downstream. However, this involves a precise, symmetric backward connectivity pattern, which is thought to be impossible in the brain. Here we demonstrate that this strong architectural constraint is not required for effective error propagation. We present a surprisingly simple mechanism that assigns blame by multiplying errors by even random synaptic weights. This mechanism can transmit teaching signals across multiple layers of neurons and performs as effectively as backpropagation on a variety of tasks. Our results help reopen questions about how the brain could use error signals and dispel long-held assumptions about algorithmic constraints on learning.

Date: 2016
References: Add references at CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
https://www.nature.com/articles/ncomms13276 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:7:y:2016:i:1:d:10.1038_ncomms13276

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/ncomms13276

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().