A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection

Manli Zhu, Edmond S. L. Ho and Hubert P. H. Shum
Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2022

A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection

Abstract

Detecting human-object interactions is essential for comprehensive understanding of visual scenes. In particular, spatial connections between humans and objects are important cues for reasoning interactions. To this end, we propose a skeleton-aware graph convolutional network for human-object interaction detection, named SGCN4HOI. Our network exploits the spatial connections between human keypoints and object keypoints to capture their fine-grained structural interactions via graph convolutions. It fuses such geometric features with visual features and spatial configuration features obtained from human-object pairs. Furthermore, to better preserve the object structural information and facilitate human-object interaction detection, we propose a novel skeleton-based object keypoints representation. The performance of SGCN4HOI is evaluated in the public benchmark V-COCO dataset. Experimental results show that the proposed approach outperforms the state-of-the-art pose-based models and achieves competitive performance against other models.


Downloads


YouTube


Cite This Research

Plain Text

Manli Zhu, Edmond S. L. Ho and Hubert P. H. Shum, "A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection," in SMC '22: Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics, pp. 275-281, Prague, Czech Republic, IEEE, Oct 2022.

BibTeX

@inproceedings{zhu22skeleton,
 author={Zhu, Manli and Ho, Edmond S. L. and Shum, Hubert P. H.},
 booktitle={Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics},
 series={SMC '22},
 title={A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection},
 year={2022},
 month={10},
 pages={275--281},
 numpages={7},
 doi={10.1109/SMC53654.2022.9945149},
 publisher={IEEE},
 location={Prague, Czech Republic},
}

RIS

TY  - CONF
AU  - Zhu, Manli
AU  - Ho, Edmond S. L.
AU  - Shum, Hubert P. H.
T2  - Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics
TI  - A Skeleton-Aware Graph Convolutional Network for Human-Object Interaction Detection
PY  - 2022
Y1  - 10 2022
SP  - 275
EP  - 281
DO  - 10.1109/SMC53654.2022.9945149
PB  - IEEE
ER  - 


Supporting Grants

Northumbria University

Postgraduate Research Scholarship (Ref: ): £65,000, Principal Investigator ()
Received from Faculty of Engineering and Environment, Northumbria University, UK, 2020-2022
Project Page

Similar Research

Manli Zhu, Edmond S. L. Ho, Shuang Chen, Longzhi Yang and Hubert P. H. Shum, "Geometric Features Enhanced Human-Object Interaction Detection", IEEE Transactions on Instrumentation and Measurement (TIM), 2024
Tanqiu Qiao, Ruochen Li, Frederick W. B. Li and Hubert P. H. Shum, "From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos", Proceedings of the 2024 International Conference on Pattern Recognition (ICPR), 2024
Tanqiu Qiao, Qianhui Men, Frederick W. B. Li, Yoshiki Kubotani, Shigeo Morishima and Hubert P. H. Shum, "Geometric Features Informed Multi-Person Human-Object Interaction Recognition in Videos", Proceedings of the 2022 European Conference on Computer Vision (ECCV), 2022
Qianhui Men, Howard Leung, Edmond S. L. Ho and Hubert P. H. Shum, "A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition", Proceedings of the 2020 International Conference on Pattern Recognition (ICPR), 2020
Ying Huang, Hubert P. H. Shum, Edmond S. L. Ho and Nauman Aslam, "High-Speed Multi-Person Pose Estimation with Deep Feature Transfer", Computer Vision and Image Understanding (CVIU), 2020

 

Last updated on 6 October 2024
RSS Feed