Accurate deep neural network inference using computational phase-change memory

Vinay Joshi, Manuel Le Gallo (), Simon Haefeli, Irem Boybat, S. R. Nandakumar, Christophe Piveteau, Martino Dazzi, Bipin Rajendran, Abu Sebastian () and Evangelos Eleftheriou
Additional contact information
Vinay Joshi: IBM Research - Zurich
Manuel Le Gallo: IBM Research - Zurich
Simon Haefeli: IBM Research - Zurich
Irem Boybat: IBM Research - Zurich
S. R. Nandakumar: IBM Research - Zurich
Christophe Piveteau: IBM Research - Zurich
Martino Dazzi: IBM Research - Zurich
Bipin Rajendran: King’s College London
Abu Sebastian: IBM Research - Zurich
Evangelos Eleftheriou: IBM Research - Zurich

Nature Communications, 2020, vol. 11, issue 1, 1-13

Abstract: Abstract In-memory computing using resistive memory devices is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to phase-change memory (PCM) devices. We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on CIFAR-10 and a top-1 accuracy of 71.6% on ImageNet benchmarks after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one-day period, where each of the 361,722 synaptic weights is programmed on just two PCM devices organized in a differential configuration.

Date: 2020
References: Add references at CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
https://www.nature.com/articles/s41467-020-16108-9 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:11:y:2020:i:1:d:10.1038_s41467-020-16108-9

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-020-16108-9

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().