A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition

Qianhui Men, Howard Leung, Edmond S. L. Ho and Hubert P. H. Shum
Proceedings of the 2020 International Conference on Pattern Recognition (ICPR), 2020

Citation: 4##

A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition
## Citation counts from Google Scholar as of 2022

Abstract

This paper addresses the problem of recognizing human-human interaction from skeletal sequences. Existing works are mainly designed to classify single human action. Many of them simply stack the movement features of two characters to deal with human interaction, while neglecting the abundant relationships between characters. In this paper, we propose a novel two-stream recurrent neural network by adopting the geometric features from both single actions and interactions to describe the spatial correlations with different discriminative abilities. The first stream is constructed under pairwise joint distance (PJD) in a fully-connected mesh to categorize the interactions with explicit distance patterns. To better distinguish similar interactions, in the second stream, we combine PJD with the spatial features from individual joint positions using graph convolutions to detect the implicit correlations among joints, where the joint connections in graph is adaptive for flexible correlations. After spatial modeling, each stream is fed to a bidirectional LSTM to encode two-way temporal properties. To take advantage of the diverse discriminative power of the two streams, we come up with a late fusion algorithm to combine their output predictions concerning information entropy. Experimental results show that the proposed framework achieves state-of-the-art performance on 3D and comparable performance on 2D interaction datasets. Moreover, the late fusion results demonstrate the effectiveness of improving the recognition accuracy compared with single streams.

Downloads

YouTube

Citations

BibTeX

@inproceedings{men20interaction,
 author={Men, Qianhui and Leung, Howard and Ho, Edmond S. L. and Shum, Hubert P. H.},
 booktitle={Proceedings of the 2020 International Conference on Pattern Recognition},
 series={ICPR '20},
 title={A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition},
 year={2020},
 month={Jan},
 pages={2771--2778},
 numpages={8},
 doi={10.1109/ICPR48806.2021.9412538},
 location={Milan, Italy},
}

RIS

TY  - CONF
AU  - Men, Qianhui
AU  - Leung, Howard
AU  - Ho, Edmond S. L.
AU  - Shum, Hubert P. H.
T2  - Proceedings of the 2020 International Conference on Pattern Recognition
TI  - A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition
PY  - 2020
Y1  - Jan 2020
SP  - 2771
EP  - 2778
DO  - 10.1109/ICPR48806.2021.9412538
ER  - 

Plain Text

Qianhui Men, Howard Leung, Edmond S. L. Ho and Hubert P. H. Shum, "A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition," in ICPR '20: Proceedings of the 2020 International Conference on Pattern Recognition, pp. 2771-2778, Milan, Italy, Jan 2020.

Similar Research

Qianhui Men, Edmond S. L. Ho, Hubert P. H. Shum and Howard Leung, "A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction", IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2021
Qianhui Men, Hubert P. H. Shum, Edmond S. L. Ho and Howard Leung, "GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interaction", Computers and Graphics (C&G), 2022
Tanqiu Qiao, Qianhui Men, Frederick W. B. Li, Yoshiki Kubotani, Shigeo Morishima and Hubert P. H. Shum, "Geometric Features Informed Multi-person Human-object Interaction Recognition in Videos", Proceedings of the 2022 European Conference on Computer Vision (ECCV), 2022

 

 
 

Last updated on 25 September 2022, RSS Feeds