Modulated Memory Network for Video Object Segmentation

Hannan Lu, Zixian Guo and Wangmeng Zuo ()
Additional contact information
Hannan Lu: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Zixian Guo: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Wangmeng Zuo: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China

Mathematics, 2024, vol. 12, issue 6, 1-17

Abstract: Existing video object segmentation (VOS) methods based on matching techniques commonly employ a reference set comprising historical segmented frames, referred to as ‘memory frames’, to facilitate the segmentation process. However, these methods suffer from the following limitations: ( i ) Inherent segmentation errors in memory frames can propagate and accumulate errors when utilized as templates for subsequent segmentation. ( ii ) The non-local matching technique employed in top-leading solutions often fails to incorporate positional information, potentially leading to incorrect matching. In this paper, we introduce the Modulated Memory Network (MMN) for VOS. Our MMN enhances matching-based VOS methods in the following ways: ( i ) Introducing an Importance Modulator, which adjusts memory frames using adaptive weight maps generated based on the segmentation confidence associated with each frame. ( ii ) Incorporating a Position Modulator that encodes spatial and temporal positional information for both memory frames and the current frame. The proposed modulator improves matching accuracy by embedding positional information. Meanwhile, the Importance Modulator mitigates error propagation and accumulation by incorporating confidence-based modulation. Through extensive experimentation, we demonstrate the effectiveness of our proposed MMN, which also achieves promising performance on VOS benchmarks.

Keywords: video object segmentation; matching based; memory network; modulator (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/6/863/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/6/863/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:6:p:863-:d:1357645

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().