Semantic Segmentation of UAV Images Based on Transformer Framework with Context Information

Satyawant Kumar, Abhishek Kumar and Dong-Gyu Lee ()
Additional contact information
Satyawant Kumar: Department of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea
Abhishek Kumar: Department of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea
Dong-Gyu Lee: Department of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea

Mathematics, 2022, vol. 10, issue 24, 1-17

Abstract: With the advances in Unmanned Aerial Vehicles (UAVs) technology, aerial images with huge variations in the appearance of objects and complex backgrounds have opened a new direction of work for researchers. The task of semantic segmentation becomes more challenging when capturing inherent features in the global and local context for UAV images. In this paper, we proposed a transformer-based encoder-decoder architecture to address this issue for the precise segmentation of UAV images. The inherent feature representation of the UAV images is exploited in the encoder network using a self-attention-based transformer framework to capture long-range global contextual information. A Token Spatial Information Fusion (TSIF) module is proposed to take advantage of a convolution mechanism that can capture local details. It fuses the local contextual details about the neighboring pixels with the encoder network and makes semantically rich feature representations. We proposed a decoder network that processes the output of the encoder network for the final semantic level prediction of each pixel. We demonstrate the effectiveness of this architecture on UAVid and Urban Drone datasets, where we achieved mIoU of 61.93% and 73.65%, respectively.

Keywords: semantic segmentation; UAV street scene images; transformer; global and local context (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/24/4735/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/24/4735/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:24:p:4735-:d:1002392

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().