A NOVEL TRANSFORMER METHOD PRETRAINED WITH MASKED AUTOENCODERS AND FRACTAL DIMENSION FOR DIABETIC RETINOPATHY CLASSIFICATION

Yaoming Yang, Zhao Zha, Chennan Zhou, Lida Zhang, Shuxia Qiu and Peng Xu
Additional contact information
Yaoming Yang: College of Science, China Jiliang University, Hangzhou 310018, P. R. China
Zhao Zha: ��Zhejiang Wandekai Fluid Equipment Technology Co., Ltd, Yuhuan 317609, P. R. China
Chennan Zhou: College of Science, China Jiliang University, Hangzhou 310018, P. R. China
Lida Zhang: College of Science, China Jiliang University, Hangzhou 310018, P. R. China
Shuxia Qiu: College of Science, China Jiliang University, Hangzhou 310018, P. R. China
Peng Xu: College of Science, China Jiliang University, Hangzhou 310018, P. R. China‡Key Laboratory of Intelligent Manufacturing Quality, Big Data Tracing and Analysis of Zhejiang Province, Hangzhou 310018, P. R. China

FRACTALS (fractals), 2024, vol. 32, issue 03, 1-11

Abstract: Diabetic retinopathy (DR) is one of the leading causes of blindness in a significant portion of the working population, and its damage on vision is irreversible. Therefore, rapid diagnosis on DR is crucial for saving the patient’s eyesight. Since Transformer shows superior performance in the field of computer vision compared with Convolutional Neural Networks (CNNs), it has been proposed and applied in computer aided diagnosis of DR. However, a large number of images should be used for training due to the lack of inductive bias in Transformers. It has been demonstrated that the retinal vessels follow self-similar fractal scaling law, and the fractal dimension of DR patients shows an evident difference from that of normal people. Based on this, the fractal dimension is introduced as a prior into Transformers to mitigate the adverse influence of lack of inductive bias on model performance. A new Transformer method pretrained with Masked Autoencoders and fractal dimension (MAEFD) is developed and proposed in this paper. The experiments on the APTOS dataset show that the classification performance for DR by the proposed MAEFD can be substantially improved. Additionally, the present model pretrained with 100,000 retinal images outperforms that pretrained with 1 million natural images in terms of DR classification performance.

Keywords: Fractal Dimension; Vision Transformer; Masked Autoencoders; Diabetic Retinopathy; Retinal Image (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0218348X24500609
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:fracta:v:32:y:2024:i:03:n:s0218348x24500609

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0218348X24500609

Access Statistics for this article

FRACTALS (fractals) is currently edited by Tara Taylor

More articles in FRACTALS (fractals) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().