In-sensor compressing via programmable optoelectronic sensors based on van der Waals heterostructures for intelligent machine vision

Haoxin Huang, Shuhui Shi, Jiajia Zha, Yunpeng Xia, Huide Wang, Peng Yang, Long Zheng, Songcen Xu, Wei Wang, Yi Ren, Yongji Wang, Ye Chen, Hau Ping Chan, Johnny C. Ho, Yang Chai (), Zhongrui Wang () and Chaoliang Tan ()
Additional contact information
Haoxin Huang: City University of Hong Kong
Shuhui Shi: University of Hong Kong
Jiajia Zha: City University of Hong Kong
Yunpeng Xia: City University of Hong Kong
Huide Wang: Shenzhen University
Peng Yang: Shenzhen Technology University
Long Zheng: The Chinese University of Hong Kong
Songcen Xu: The Hong Kong University of Science and Technology
Wei Wang: City University of Hong Kong
Yi Ren: City University of Hong Kong
Yongji Wang: City University of Hong Kong
Ye Chen: The Chinese University of Hong Kong
Hau Ping Chan: City University of Hong Kong
Johnny C. Ho: City University of Hong Kong
Yang Chai: The Hong Kong Polytechnic University
Zhongrui Wang: Southern University of Science and Technology
Chaoliang Tan: City University of Hong Kong

Nature Communications, 2025, vol. 16, issue 1, 1-13

Abstract: Abstract Efficiently capturing multidimensional signals containing spectral and temporal information is crucial for intelligent machine vision. Although in-sensor computing shows promise for efficient visual processing by reducing data transfer, its capability to compress temporal/spectral data is rarely reported. Here we demonstrate a programmable two-dimensional (2D) heterostructure-based optoelectronic sensor integrating sensing, memory, and computation for in-sensor data compression. Our 2D sensor captured and memorized/encoded optical signals, leading to in-device snapshot compression of dynamic videos and three-dimensional spectral data with a compression ratio of 8:1. The reconstruction quality, indicated by a peak signal-to-noise ratio value of 15.81 dB, is comparable to the 16.21 dB achieved through software. Meanwhile, the compressed action videos (in the form of 2D images) preserve all semantic information and can be accurately classified using in-sensor convolution without decompression, achieving accuracy on par with uncompressed videos (93.18% vs 83.43%). Our 2D optoelectronic sensors promote the development of efficient intelligent vision systems at the edge.

Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41467-025-59104-7 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-59104-7

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-025-59104-7

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().