Ultrafast machine vision with 2D material neural network image sensors

Lukas Mennel (), Joanna Symonowicz, Stefan Wachter, Dmitry K. Polyushkin, Aday J. Molina-Mendoza and Thomas Mueller ()
Additional contact information
Lukas Mennel: Vienna University of Technology
Joanna Symonowicz: Vienna University of Technology
Stefan Wachter: Vienna University of Technology
Dmitry K. Polyushkin: Vienna University of Technology
Aday J. Molina-Mendoza: Vienna University of Technology
Thomas Mueller: Vienna University of Technology

Nature, 2020, vol. 579, issue 7797, 62-66

Abstract: Abstract Machine vision technology has taken huge leaps in recent years, and is now becoming an integral part of various intelligent systems, including autonomous vehicles and robotics. Usually, visual information is captured by a frame-based camera, converted into a digital format and processed afterwards using a machine-learning algorithm such as an artificial neural network (ANN)1. The large amount of (mostly redundant) data passed through the entire signal chain, however, results in low frame rates and high power consumption. Various visual data preprocessing techniques have thus been developed2–7 to increase the efficiency of the subsequent signal processing in an ANN. Here we demonstrate that an image sensor can itself constitute an ANN that can simultaneously sense and process optical images without latency. Our device is based on a reconfigurable two-dimensional (2D) semiconductor8,9 photodiode10–12 array, and the synaptic weights of the network are stored in a continuously tunable photoresponsivity matrix. We demonstrate both supervised and unsupervised learning and train the sensor to classify and encode images that are optically projected onto the chip with a throughput of 20 million bins per second.

Date: 2020
References: Add references at CitEc
Citations: View citations in EconPapers (36)

Downloads: (external link)
https://www.nature.com/articles/s41586-020-2038-x Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:nature:v:579:y:2020:i:7797:d:10.1038_s41586-020-2038-x

Ordering information: This journal article can be ordered from
https://www.nature.com/

DOI: 10.1038/s41586-020-2038-x

Access Statistics for this article

Nature is currently edited by Magdalena Skipper

More articles in Nature from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().