Kinect Posture Reconstruction based on a Local Mixture of Gaussian Process Models

Zhiguang Liu, Liuyang Zhou, Howard Leung and Hubert P. H. Shum
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2016

Impact Factor: 5.226# Citation: 54## REF 2021 Submission

Kinect Posture Reconstruction based on a Local Mixture of Gaussian Process Models
# Impact factors from the Journal Citation Reports 2021
## Citation counts from Google Scholar as of 2022

Abstract

Depth sensor based 3D human motion estimation hardware such as Kinect has made interactive applications more popular recently. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. In this paper, we propose a new real-time probabilistic framework to enhance the accuracy of live captured postures that belong to one of the action classes in the database. We adopt the Gaussian Process model as a prior to leverage the position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the accurate parts of the observed posture, we embed a set of joint reliability measurements into the optimization framework. A major drawback of Gaussian Process is its cubic learning complexity when dealing with a large database due to the inverse of a covariance matrix. To solve the problem, we propose a new method based on a local mixture of Gaussian Processes, in which Gaussian Processes are defined in local regions of the state space. Due to the significantly decreased sample size in each local Gaussian Process, the learning time is greatly reduced. At the same time, the prediction speed is enhanced as the weighted mean prediction for a given sample is determined by the nearby local models only. Our system also allows incrementally updating a specific local Gaussian Process in real time, which enhances the likelihood of adapting to run-time postures that are different from those in the database. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time applications such as motion-based gaming and sport training.

Downloads

YouTube

Citations

BibTeX

@article{liu16kinect,
 author={Liu, Zhiguang and Zhou, Liuyang and Leung, Howard and Shum, Hubert P. H.},
 journal={IEEE Transactions on Visualization and Computer Graphics},
 title={Kinect Posture Reconstruction based on a Local Mixture of Gaussian Process Models},
 year={2016},
 month={Nov},
 volume={22},
 number={11},
 pages={2437-2450},
 numpages={14},
 doi={10.1109/TVCG.2015.2510000},
 issn={1077-2626},
 publisher={IEEE},
}

RIS

TY  - JOUR
AU  - Liu, Zhiguang
AU  - Zhou, Liuyang
AU  - Leung, Howard
AU  - Shum, Hubert P. H.
T2  - IEEE Transactions on Visualization and Computer Graphics
TI  - Kinect Posture Reconstruction based on a Local Mixture of Gaussian Process Models
PY  - 2016
Y1  - Nov 2016
VL  - 22
IS  - 11
SP  - 2437-2450
EP  - 2437-2450
DO  - 10.1109/TVCG.2015.2510000
SN  - 1077-2626
PB  - IEEE
ER  - 

Plain Text

Zhiguang Liu, Liuyang Zhou, Howard Leung and Hubert P. H. Shum, "Kinect Posture Reconstruction based on a Local Mixture of Gaussian Process Models," IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 11, pp. 2437-2450, IEEE, Nov 2016.

Similar Research

Liuyang Zhou, Zhiguang Liu, Howard Leung and Hubert P. H. Shum, "Posture Reconstruction Using Kinect with a Probabilistic Model", Proceedings of the 2014 ACM Symposium on Virtual Reality Software and Technology (VRST), 2014
Hubert P. H. Shum, Edmond S. L. Ho, Yang Jiang and Shu Takagi, "Real-Time Posture Reconstruction for Microsoft Kinect", IEEE Transactions on Cybernetics (TCyb), 2013
Hubert P. H. Shum and Edmond S. L. Ho, "Real-time Physical Modelling of Character Movements with Microsoft Kinect", Proceedings of the 2012 ACM Symposium on Virtual Reality Software and Technology (VRST), 2012
Pierre Plantard, Hubert P. H. Shum and Franck Multon, "Filtered Pose Graph for Efficient Kinect Pose Reconstruction", Multimedia Tools and Applications (MTAP), 2017
Hubert P. H. Shum, "Serious Games with Human-Object Interactions using RGB-D Camera", Proceedings of the 2013 International Conference on Motion in Games (MIG) Posters, 2013
Pierre Plantard, Antoine Muller, Charles Pontonnier, Georges Dumont, Hubert P. H. Shum and Franck Multon, "Inverse Dynamics Based on Occlusion-resistant Kinect Data: Is It Usable for Ergonomics?", International Journal of Industrial Ergonomics (IJIE), 2017

 

 
 

Last updated on 25 September 2022, RSS Feeds