A Frequency Attention-Based Dual-Stream Network for Image Inpainting Forensics

Hongquan Wang, Xinshan Zhu (), Chao Ren, Lan Zhang and Shugen Ma
Additional contact information
Hongquan Wang: School of Electrical and Information Engineering, Tianjin University, Tianjing 300072, China
Xinshan Zhu: School of Electrical and Information Engineering, Tianjin University, Tianjing 300072, China
Chao Ren: School of Electrical and Information Engineering, Tianjin University, Tianjing 300072, China
Lan Zhang: School of Electrical and Information Engineering, Tianjin University, Tianjing 300072, China
Shugen Ma: School of Electrical and Information Engineering, Tianjin University, Tianjing 300072, China

Mathematics, 2023, vol. 11, issue 12, 1-23

Abstract: The rapid development of digital image inpainting technology is causing serious hidden danger to the security of multimedia information. In this paper, a deep network called frequency attention-based dual-stream network (FADS-Net) is proposed for locating the inpainting region. FADS-Net is established by a dual-stream encoder and an attention-based blue-associative decoder. The dual-stream encoder includes two feature extraction streams, the raw input stream (RIS) and the frequency recalibration stream (FRS). RIS directly captures feature maps from the raw input, while FRS performs feature extraction after recalibrating the input via learning in the frequency domain. In addition, a module based on dense connection is designed to ensure efficient extraction and full fusion of dual-stream features. The attention-based associative decoder consists of a main decoder and two branch decoders. The main decoder performs up-sampling and fine-tuning of fused features by using attention mechanisms and skip connections, and ultimately generates the predicted mask for the inpainted image. Then, two branch decoders are utilized to further supervise the training of two feature streams, ensuring that they both work effectively. A joint loss function is designed to supervise the training of the entire network and two feature extraction streams for ensuring optimal forensic performance. Extensive experimental results demonstrate that the proposed FADS-Net achieves superior localization accuracy and robustness on multiple datasets compared to the state-of-the-art inpainting forensics methods.

Keywords: inpainting forensics; deep convolutional neural network; learning in frequency domain; dual-stream feature extraction; attention mechanism (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/12/2593/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/12/2593/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:12:p:2593-:d:1164983

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().