Real-Time Posture Reconstruction for Microsoft Kinect

Hubert P. H. Shum, Edmond S. L. Ho, Yang Jiang and Shu Takagi
IEEE Transactions on Cybernetics (TCyb), 2013

REF 2014 Submitted Output Impact Factor: 9.4 Top 10% Journal in Computer Science, Artificial Intelligence Citation: 181#

Real-Time Posture Reconstruction for Microsoft Kinect
# According to Google Scholar 2024


The recent advancement of motion recognition using Microsoft Kinect stimulates many new ideas in motion capture and virtual reality applications. Utilizing a pattern recognition algorithm, Kinect can determine the positions of different body parts from the user. However, due to the use of a single depth camera, recognition accuracy drops significantly when the parts are occluded. This hugely limits the usability of applications that involves interaction with external objects, such as sport training or exercising systems. The problem becomes more critical when Kinect incorrectly perceives the body parts. This is because applications have limited information about the recognition correctness, and using those parts to synthesize a body postures would result in serious visual artifacts. In this paper, we propose a new method to reconstruct valid movement from incomplete and noisy postures captured by Kinect. We first design a set of measurements that objectively evaluates the degree of reliability on each tracked body part. By incorporating the reliability estimation into a motion database query during run-time, we obtain a set of similar postures that are kinematically valid. These postures are used to construct a latent space, which is known as the natural posture space in our system, with local Principle Component Analysis (PCA). We finally apply frame-based optimization in the space to synthesize a new posture that closely resembles the true user posture while satisfying kinematic constraints. Experimental results show that our method can significantly improve the quality of the recognized posture under severely occluded environments, such as a person exercising with a basketball or moving in a small room.





 author={Shum, Hubert P. H. and Ho, Edmond S. L. and Jiang, Yang and Takagi, Shu},
 journal={IEEE Transactions on Cybernetics},
 series={TCyb '24},
 title={Real-Time Posture Reconstruction for Microsoft Kinect},


AU  - Shum, Hubert P. H.
AU  - Ho, Edmond S. L.
AU  - Jiang, Yang
AU  - Takagi, Shu
T2  - IEEE Transactions on Cybernetics
TI  - Real-Time Posture Reconstruction for Microsoft Kinect
PY  - 2013
VL  - 43
IS  - 5
SP  - 1357
EP  - 1369
DO  - 10.1109/TCYB.2013.2275945
SN  - 2168-2267
ER  - 

Plain Text

Hubert P. H. Shum, Edmond S. L. Ho, Yang Jiang and Shu Takagi, "Real-Time Posture Reconstruction for Microsoft Kinect," IEEE Transactions on Cybernetics, vol. 43, no. 5, pp. 1357-1369, IEEE, 2013.

Supporting Grants

Northumbria University

Start-Up Research Grant (Ref: 100700-IO114845): £2,000, Principal Investigator ()
Received from Faculty of Engineering and Environment, Northumbria University, UK, 2012-2013
Project Page

Similar Research

Zhiguang Liu, Liuyang Zhou, Howard Leung and Hubert P. H. Shum, "Kinect Posture Reconstruction Based on a Local Mixture of Gaussian Process Models", IEEE Transactions on Visualization and Computer Graphics (TVCG), 2016
Liuyang Zhou, Zhiguang Liu, Howard Leung and Hubert P. H. Shum, "Posture Reconstruction Using Kinect with a Probabilistic Model", Proceedings of the 2014 ACM Symposium on Virtual Reality Software and Technology (VRST), 2014
Pierre Plantard, Hubert P. H. Shum and Franck Multon, "Filtered Pose Graph for Efficient Kinect Pose Reconstruction", Multimedia Tools and Applications (MTAP), 2017
Hubert P. H. Shum, "Serious Games with Human-Object Interactions using RGB-D Camera", Proceedings of the 2013 ACM International Conference on Motion in Games (MIG) Posters, 2013
Hubert P. H. Shum and Edmond S. L. Ho, "Real-Time Physical Modelling of Character Movements with Microsoft Kinect", Proceedings of the 2012 ACM Symposium on Virtual Reality Software and Technology (VRST), 2012



Last updated on 15 July 2024
RSS Feed