A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection

Manli Zhu, Edmond S. L. Ho and Hubert P. H. Shum
Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2022

A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection

Abstract

Detecting human-object interactions is essential for comprehensive understanding of visual scenes. In particular, spatial connections between humans and objects are important cues for reasoning interactions. To this end, we propose a skeleton-aware graph convolutional network for human-object interaction detection, named SGCN4HOI. Our network exploits the spatial connections between human keypoints and object keypoints to capture their fine-grained structural interactions via graph convolutions. It fuses such geometric features with visual features and spatial configuration features obtained from human-object pairs. Furthermore, to better preserve the object structural information and facilitate human-object interaction detection, we propose a novel skeleton-based object keypoints representation. The performance of SGCN4HOI is evaluated in the public benchmark V-COCO dataset. Experimental results show that the proposed approach outperforms the state-of-the-art pose-based models and achieves competitive performance against other models.

Downloads

YouTube

Citations

BibTeX

@inproceedings{zhu22skeleton,
 author={Zhu, Manli and Ho, Edmond S. L. and Shum, Hubert P. H.},
 booktitle={Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics},
 series={SMC '22},
 title={A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection},
 year={2022},
 month={10},
 pages={275--281},
 numpages={7},
 doi={10.1109/SMC53654.2022.9945149},
 publisher={IEEE},
 location={Prague, Czech Republic},
}

RIS

TY  - CONF
AU  - Zhu, Manli
AU  - Ho, Edmond S. L.
AU  - Shum, Hubert P. H.
T2  - Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics
TI  - A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection
PY  - 2022
Y1  - 10 2022
SP  - 275
EP  - 281
DO  - 10.1109/SMC53654.2022.9945149
PB  - IEEE
ER  - 

Plain Text

Manli Zhu, Edmond S. L. Ho and Hubert P. H. Shum, "A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection," in SMC '22: Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics, pp. 275-281, Prague, Czech Republic, IEEE, Oct 2022.

Supporting Grants

Northumbria University

Postgraduate Research Scholarship (Ref: ): £65,000, Principal Investigator ()
Received from Faculty of Engineering and Environment, Northumbria University, UK, 2020-2022
Project Page

Similar Research

Tanqiu Qiao, Qianhui Men, Frederick W. B. Li, Yoshiki Kubotani, Shigeo Morishima and Hubert P. H. Shum, "Geometric Features Informed Multi-Person Human-Object Interaction Recognition in Videos", Proceedings of the 2022 European Conference on Computer Vision (ECCV), 2022
Qianhui Men, Howard Leung, Edmond S. L. Ho and Hubert P. H. Shum, "A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition", Proceedings of the 2020 International Conference on Pattern Recognition (ICPR), 2020
Ying Huang, Hubert P. H. Shum, Edmond S. L. Ho and Nauman Aslam, "High-Speed Multi-Person Pose Estimation with Deep Feature Transfer", Computer Vision and Image Understanding (CVIU), 2020
Yijun Shen, Longzhi Yang, Edmond S. L. Ho and Hubert P. H. Shum, "Interaction-Based Human Activity Comparison", IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020

 

 

Last updated on 25 March 2024
RSS Feed