Enhancing diagnostic accuracy in rare and common fundus diseases with a knowledge-rich vision-language model

Meng Wang, Tian Lin, Aidi Lin, Kai Yu, Yuanyuan Peng, Lianyu Wang, Cheng Chen, Ke Zou, Huiyu Liang, Man Chen, Xue Yao, Meiqin Zhang, Binwei Huang, Chaoxin Zheng, Peixin Zhang, Wei Chen, Yilong Luo, Yifan Chen, Honghe Xia, Tingkun Shi, Qi Zhang, Jinming Guo, Xiaolin Chen, Jingcheng Wang, Yih Chung Tham, Dianbo Liu, Wendy Wong, Sahil Thakur, Beau J. Fenner, Danqi Fang, Siying Liu, Qingyun Liu, Yuqiang Huang, Hongqiang Zeng, Yanda Meng, Yukun Zhou, Zehua Jiang, Minghui Qiu, Changqing Zhang, Xinjian Chen, Sophia Y. Wang, Cecilia S. Lee, Lucia Sobrin, Carol Y. Cheung, Chi Pui Pang, Pearse A. Keane, Ching-Yu Cheng (), Haoyu Chen () and Huazhu Fu ()
Additional contact information
Meng Wang: National University of Singapore
Tian Lin: Shantou University and the Chinese University of Hong Kong
Aidi Lin: Shantou University and the Chinese University of Hong Kong
Kai Yu: University of Pennsylvania
Yuanyuan Peng: Anhui Medical University
Lianyu Wang: Nanjing University of Aeronautics and Astronautics
Cheng Chen: Department of Electrical and Electronic Engineering, The University of Hong Kong
Ke Zou: Sichuan University
Huiyu Liang: Shantou University and the Chinese University of Hong Kong
Man Chen: Shantou University and the Chinese University of Hong Kong
Xue Yao: Shantou University and the Chinese University of Hong Kong
Meiqin Zhang: Shantou University and the Chinese University of Hong Kong
Binwei Huang: Shantou University and the Chinese University of Hong Kong
Chaoxin Zheng: Shantou University and the Chinese University of Hong Kong
Peixin Zhang: Shantou University and the Chinese University of Hong Kong
Wei Chen: Shantou University and the Chinese University of Hong Kong
Yilong Luo: Shantou University and the Chinese University of Hong Kong
Yifan Chen: Shantou University and the Chinese University of Hong Kong
Honghe Xia: Shantou University and the Chinese University of Hong Kong
Tingkun Shi: Shantou University and the Chinese University of Hong Kong
Qi Zhang: Shantou University and the Chinese University of Hong Kong
Jinming Guo: Shantou University and the Chinese University of Hong Kong
Xiaolin Chen: Shantou University and the Chinese University of Hong Kong
Jingcheng Wang: Big Vision Medical Technology Ltd.
Yih Chung Tham: National University of Singapore
Dianbo Liu: National University of Singapore
Wendy Wong: National University of Singapore
Sahil Thakur: Singapore National Eye Centre
Beau J. Fenner: Singapore National Eye Centre
Danqi Fang: The Chinese University of Hong Kong
Siying Liu: Shenzhen Longgang E.N.T Hospital
Qingyun Liu: Shenzhen Longgang E.N.T Hospital
Yuqiang Huang: Shantou University and the Chinese University of Hong Kong
Hongqiang Zeng: Dongguan Songshan Lake Central Hospital
Yanda Meng: University of Exeter
Yukun Zhou: University College London
Zehua Jiang: Tsinghua Medicine of Tsinghua University
Minghui Qiu: Foshan Aier Zhuoyue Eye Hospital
Changqing Zhang: Tianjin University
Xinjian Chen: Soochow University
Sophia Y. Wang: Stanford University School of Medicine
Cecilia S. Lee: University of Washington
Lucia Sobrin: Harvard Medical School
Carol Y. Cheung: The Chinese University of Hong Kong
Chi Pui Pang: Shantou University and the Chinese University of Hong Kong
Pearse A. Keane: NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust
Ching-Yu Cheng: National University of Singapore
Haoyu Chen: Shantou University and the Chinese University of Hong Kong
Huazhu Fu: Technology and Research (A*STAR)

Nature Communications, 2025, vol. 16, issue 1, 1-17

Abstract: Abstract Previous foundation models for fundus images were pre-trained with limited disease categories and knowledge base. Here we introduce RetiZero, a vision-language model that incorporates knowledge from over 400 fundus diseases. The model is pre-trained on 341,896 fundus images with accompanying text descriptions gathered from diverse sources across multiple ethnicities and countries. RetiZero demonstrates exceptional performance across various downstream tasks including zero-shot disease recognition, image-to-image retrieval, clinical diagnosis assistance, few-shot fine-tuning, and cross-domain disease identification. In zero-shot scenarios, it achieves Top-5 accuracies of 0.843 for 15 diseases and 0.756 for 52 diseases, while for image-to-image retrieval, it scores 0.950 and 0.886 respectively. Notably, RetiZero’s Top-3 zero-shot performance exceeds the average diagnostic accuracy of 19 ophthalmologists from Singapore, China, and the United States. The model particularly enhances clinicians’ ability to diagnose rare fundus conditions, highlighting its potential value for integration into clinical settings where diverse eye diseases are encountered.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41467-025-60577-9 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-60577-9

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-025-60577-9

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().