Universiti Teknologi Malaysia Institutional Repository

Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection

Zhang, Jiali and Mohd. Yunos, Zuriahati and Haron, Habibollah (2023) Interactivity recognition graph neural network (IR-GNN) model for improving human-object interaction detection. Electronics (Switzerland), 12 (2). pp. 1-19. ISSN 2079-9292

[img] PDF
3MB

Official URL: http://dx.doi.org/10.3390/electronics12020470

Abstract

Human–object interaction (HOI) detection is important for promoting the development of many fields such as human–computer interactions, service robotics, and video security surveillance. A high percentage of human–object pairs with invalid interactions are discovered in the object detection phase of conventional human–object interaction detection algorithms, resulting in inaccurate interaction detection. To recognize invalid human–object interaction pairs, this paper proposes a model structure, the interactivity recognition graph neural network (IR-GNN) model, which can directly infer the probability of human–object interactions from a graph model architecture. The model consists of three modules: The first one is the human posture feature module, which uses key points of the human body to construct relative spatial pose features and further facilitates the discrimination of human–object interactivity through human pose information. Second, a human–object interactivity graph module is proposed. The spatial relationship of human–object distance is used as the initialization weight of edges, and the graph is updated by combining the message passing of attention mechanism so that edges with interacting node pairs obtain higher weights. Thirdly, the classification module is proposed, by finally using a fully connected neural network, the interactivity of human–object pairs is binarily classified. These three modules work in collaboration to enable the effective inference of interactive possibilities. On the datasets HICO-DET and V-COCO, comparative and ablation experiments are carried out. It has been proved that our technology can improve the detection of human–object interactions.

Item Type:Article
Uncontrolled Keywords:graph neural network, human–object interaction, interactivity recognition
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:Computing
ID Code:106602
Deposited By: Widya Wahid
Deposited On:14 Jul 2024 09:19
Last Modified:14 Jul 2024 09:19

Repository Staff Only: item control page