HOME: 3D Human–Object Mesh Topology-Enhanced Interaction Recognition in Images

Weilong Peng, Cong Li, Keke Tang (), Xianyong Liu () and Meie Fang
Additional contact information
Weilong Peng: School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 511442, China
Cong Li: School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 511442, China
Keke Tang: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou 511442, China
Xianyong Liu: Robotics Institute, Ningbo University of Technology, Ningbo 315100, China
Meie Fang: School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 511442, China

Mathematics, 2022, vol. 10, issue 16, 1-15

Abstract: Human–object interaction (HOI) recognition is a very challenging task due to the ambiguity brought by occlusions, viewpoints, and poses. Because of the limited interaction information in the image domain, extracting 3D features of a point cloud has been an important means to improve the recognition performance of HOI. However, the features neglect topological features of adjacent points at low level, and the deep topology relation between a human and an object at high level. In this paper, we present a 3D human–object mesh topology enhanced method (HOME) for HOI recognition in images. In the method, human–object mesh (HOM) is built by integrating the reconstructed human and object mesh from images firstly. Therefore, under the assumption that the interaction comes from the macroscopic pattern constructed by spatial position and microscopic topology of human–object, HOM is inputted into MeshCNN to extract the effective edge features by edge-based convolution from bottom to up, as the topological features that encode the invariance of the interaction relationship. At last, topological cues are fused with visual cues to enhance the recognition performance greatly. In the experiment, HOI recognition results have achieved an improvement of about 4.3% mean average precision (mAP) in the Rare cases of the HICO-DET dataset, which verifies the effectiveness of the proposed method.

Keywords: mesh topology; interaction recognition; HOME; HOI (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/16/2841/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/16/2841/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:16:p:2841-:d:884265

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().