A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction

Qianhui Men, Edmond S. L. Ho, Hubert P. H. Shum and Howard Leung
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2021

Impact Factor: 5.859# Citation: 7##

A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction
# Impact factors from the Journal Citation Reports 2021
## Citation counts from Google Scholar as of 2022

Abstract

Recurrent neural network (RNN) has become popular for human motion prediction thanks to its ability to capture temporal dependencies. However, it has limited capacity in modeling the complex spatial relationship in the human skeletal structure. In this work, we present a novel diffusion convolutional recurrent predictor for spatial and temporal movement forecasting, with multi-step random walks traversing bidirectionally along an adaptive graph to model interdependency among body joints. In the temporal domain, existing methods rely on a single forward predictor with the produced motion deflecting to the drift route, which leads to error accumulations over time. We propose to supplement the forward predictor with a forward discriminator to alleviate such motion drift in the long term under adversarial training. The solution is further enhanced by a backward predictor and a backward discriminator to effectively reduce the error, such that the system can also look into the past to improve the prediction at early frames. The two-way spatial diffusion convolutions and two-way temporal predictors together form a quadruple network. Furthermore, we train our framework by modeling the velocity from observed motion dynamics instead of static poses to predict future movements that effectively reduces the discontinuity problem at early prediction. Our method outperforms the state of the arts on both 3D and 2D datasets, including the Human3.6M, CMU Motion Capture and Penn Action datasets. The results also show that our method correctly predicts both high-dynamic and low-dynamic moving trends with less motion drift.

Downloads

YouTube

Citations

BibTeX

@article{men21quadruple,
 author={Men, Qianhui and Ho, Edmond S. L. and Shum, Hubert P. H. and Leung, Howard},
 journal={IEEE Transactions on Circuits and Systems for Video Technology},
 title={A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction},
 year={2021},
 volume={31},
 number={9},
 pages={3417--3432},
 numpages={16},
 doi={10.1109/TCSVT.2020.3038145},
 issn={1051-8215},
 publisher={IEEE},
}

RIS

TY  - JOUR
AU  - Men, Qianhui
AU  - Ho, Edmond S. L.
AU  - Shum, Hubert P. H.
AU  - Leung, Howard
T2  - IEEE Transactions on Circuits and Systems for Video Technology
TI  - A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction
PY  - 2021
VL  - 31
IS  - 9
SP  - 3417
EP  - 3432
DO  - 10.1109/TCSVT.2020.3038145
SN  - 1051-8215
PB  - IEEE
ER  - 

Plain Text

Qianhui Men, Edmond S. L. Ho, Hubert P. H. Shum and Howard Leung, "A Quadruple Diffusion Convolutional Recurrent Network for Human Motion Prediction," IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 9, pp. 3417-3432, IEEE, 2021.

Similar Research

Qianhui Men, Hubert P. H. Shum, Edmond S. L. Ho and Howard Leung, "GAN-based Reactive Motion Synthesis with Class-aware Discriminators for Human-human Interaction", Computers and Graphics (C&G), 2022
Qianhui Men, Howard Leung, Edmond S. L. Ho and Hubert P. H. Shum, "A Two-Stream Recurrent Network for Skeleton-Based Human Interaction Recognition", Proceedings of the 2020 International Conference on Pattern Recognition (ICPR), 2020
He Wang, Edmond S. L. Ho, Hubert P. H. Shum and Zhanxing Zhu, "Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling", IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021

 

 
 

Last updated on 2 November 2022, RSS Feeds