Spectral convolutional neural network chip for in-sensor edge computing of incoherent natural light

Kaiyu Cui (), Shijie Rao, Sheng Xu, Yidong Huang (), Xusheng Cai, Zhilei Huang, Yu Wang, Xue Feng, Fang Liu, Wei Zhang, Yali Li and Shengjin Wang
Additional contact information
Kaiyu Cui: Tsinghua University
Shijie Rao: Tsinghua University
Sheng Xu: Tsinghua University
Yidong Huang: Tsinghua University
Xusheng Cai: Beijing Seetrum Technology Co.
Zhilei Huang: Beijing Seetrum Technology Co.
Yu Wang: Beijing Seetrum Technology Co.
Xue Feng: Tsinghua University
Fang Liu: Tsinghua University
Wei Zhang: Tsinghua University
Yali Li: Tsinghua University
Shengjin Wang: Tsinghua University

Nature Communications, 2025, vol. 16, issue 1, 1-10

Abstract: Abstract Optical neural networks are considered next-generation physical implementations of artificial neural networks, but their capabilities are limited by on-chip integration scale and requirement for coherent light sources. This study proposes a spectral convolutional neural network (SCNN) with matter meta-imaging. The optical convolutional layer is implemented by integrating very large-scale and pixel-aligned spectral filters on CMOS image sensor. It facilitates highly parallel spectral vector-inner products of incident incoherent natural light i.e., the direct information carrier, which empowers in-sensor optical analog computing at extremely high energy efficiency. To the best of our knowledge, this is the first integrated optical computing utilizing natural light. We employ the same SCNN chip for completely different real-world complex tasks and achieve accuracies of over 96% for pathological diagnosis and almost 100% for face anti-spoofing at video rates. These results indicate a feasible and scalable in-sensor edge computing chip of natural light for various portable terminals.

Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41467-024-55558-3 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-024-55558-3

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-024-55558-3

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().