A Visually Inspired Computational Model for Recognition of Optic Flow

Xiumin Li (), Wanyan Lin, Hao Yi, Lei Wang and Jiawei Chen
Additional contact information
Xiumin Li: College of Automation, Chongqing University, Chongqing 400030, China
Wanyan Lin: College of Automation, Chongqing University, Chongqing 400030, China
Hao Yi: Huawei Technologies Co., Ltd., Shenzhen 518129, China
Lei Wang: College of Automation, Chongqing University, Chongqing 400030, China
Jiawei Chen: College of Automation, Chongqing University, Chongqing 400030, China

Mathematics, 2023, vol. 11, issue 23, 1-13

Abstract: Foundation models trained on vast quantities of data have demonstrated impressive performance in capturing complex nonlinear relationships and accurately predicting neuronal responses. Due to the fact that deep learning neural networks depend on massive amounts of data samples and high energy consumption, foundation models based on spiking neural networks (SNNs) have the potential to significantly reduce calculation costs by training on neuromorphic hardware. In this paper, a visually inspired computational model composed of an SNN and echo state network (ESN) is proposed for the recognition of optic flow. The visually inspired SNN model serves as a foundation model that is trained using spike-timing-dependent plasticity (STDP) for extracting core features. The ESN model makes readout decisions for recognition tasks using the linear regression method. The results show that STDP can perform similar functions as non-negative matrix decomposition (NMF), i.e., generating sparse and linear superimposed readouts based on basis flow fields. Once the foundation model is fully trained from enough input samples, it can considerably reduce the training samples required for ESN readout learning. Our proposed SNN-based foundation model facilitates efficient and cost-effective task learning and could also be adapted to new stimuli that are not included in the training of the foundation model. Moreover, compared with the NMF algorithm, the foundation model trained using STDP does not need to be retrained during the testing procedure, contributing to a more efficient computational performance.

Keywords: foundation model; MT-MSTd; STDP; SNN; optic flow (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/23/4777/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/23/4777/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:23:p:4777-:d:1288261

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().