Hierarchical Graph Convolutional Networks for Action Quality Assessment

Kanglei Zhou, Yue Ma, Hubert P. H. Shum and Xiaohui Liang
IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2023

 Impact Factor: 8.4 Top 10% Journal in Engineering, Electrical & Electronic

Hierarchical Graph Convolutional Networks for Action Quality Assessment

Abstract

Action quality assessment (AQA) automatically evaluates how well humans perform actions in a given video, a technique widely used in fields such as rehabilitation medicine, athletic competitions, and specific skills assessment. However, existing works that uniformly divide the video sequence into small clips of equal length suffer from intra-clip confusion and inter-clip incoherence, hindering the further development of AQA. To address this issue, we propose a hierarchical graph convolutional network (GCN). First, semantic information confusion is corrected through clip refinement, generating the 'shot' as the basic action unit. We then construct a scene graph by combining several consecutive shots into meaningful scenes to capture local dynamics. These scenes can be viewed as different procedures of a given action, providing valuable assessment cues. The video-level representation is finally extracted via sequential action aggregation among scenes to regress the predicted score distribution, enhancing discriminative features and improving assessment performance. Experiments on the AQA-7, MTLAQA, and JIGSAWS datasets demonstrate the superiority of the proposed hierarchical GCN over state-of-the-art methods.

Downloads

YouTube

Citations

BibTeX

@article{zhou23hierarchical,
 author={Zhou, Kanglei and Ma, Yue and Shum, Hubert P. H. and Liang, Xiaohui},
 journal={IEEE Transactions on Circuits and Systems for Video Technology},
 title={Hierarchical Graph Convolutional Networks for Action Quality Assessment},
 year={2023},
 volume={33},
 number={12},
 pages={7749-7763},
 numpages={15},
 doi={10.1109/TCSVT.2023.3281413},
 issn={1051-8215},
 publisher={IEEE},
}

RIS

TY  - JOUR
AU  - Zhou, Kanglei
AU  - Ma, Yue
AU  - Shum, Hubert P. H.
AU  - Liang, Xiaohui
T2  - IEEE Transactions on Circuits and Systems for Video Technology
TI  - Hierarchical Graph Convolutional Networks for Action Quality Assessment
PY  - 2023
VL  - 33
IS  - 12
SP  - 7749-7763
EP  - 7749-7763
DO  - 10.1109/TCSVT.2023.3281413
SN  - 1051-8215
PB  - IEEE
ER  - 

Plain Text

Kanglei Zhou, Yue Ma, Hubert P. H. Shum and Xiaohui Liang, "Hierarchical Graph Convolutional Networks for Action Quality Assessment," IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 12, pp. 7749-7763, IEEE, 2023.

Supporting Grants

Similar Research

Kanglei Zhou, Ruizhi Cai, Yue Ma, Qingqing Tan, Xinning Wang, Jianguo Li, Hubert P. H. Shum, Frederick W. B. Li, Song Jin and Xiaohui Liang, "A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis", IEEE Transactions on Visualization and Computer Graphics (TVCG) - Proceedings of the 2023 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2023
Yijun Shen, He Wang, Edmond S. L. Ho, Longzhi Yang and Hubert P. H. Shum, "Posture-Based and Action-Based Graphs for Boxing Skill Visualization", Computers and Graphics (C&G), 2017
Hubert P. H. Shum, He Wang, Edmond S. L. Ho and Taku Komura, "SkillVis: A Visualization Tool for Boxing Skill Assessment", Proceedings of the 2016 ACM International Conference on Motion in Games (MIG), 2016
Hubert P. H. Shum, Taku Komura and Akinori Nagano, "Automatic Evaluation of Boxing Techniques from Captured Shadow Boxing Data", Proceedings of the 2007 Congress of International Society of Biomechanics (ISB), 2007

 

 

Last updated on 4 June 2024
RSS Feed