Occlusion for 3D Object Manipulation with Hands in Augmented Reality

Qi Feng, Hubert P. H. Shum and Shigeo Morishima
Proceedings of the 2018 Meeting on Image Recognition and Understanding (MIRU), 2018

Occlusion for 3D Object Manipulation with Hands in Augmented Reality

Abstract

By dynamically enhancing the reality with a wide range of applications, augmented reality has been drawing growing attention during recent years. Due to the need to interact with virtual objects, hand-object interaction has become a critical element in these applications. By enhancing existing objects with virtual visual appearance, tactile feedback can be provided to the user as well. However, to deal with problems of augmenting real objects with virtual data, a correct visual representation is needed. This becomes even more critical when the interaction between virtual and real objects happens. Incorrect occlusion information can be fatal to an immersive augmented reality experience. In this paper, we propose an innovative approach to handle the occlusion of augmented 3D object manipulation with hands in real-time by exploiting the nature of hand pose combined with tracking-based and model-based methods for estimating occlusion, to achieve a complete augmented reality experience without the necessities of heavy computations, complex manual segmentation processes or wearing special gloves.

Downloads

YouTube

Citations

BibTeX

@inproceedings{feng18occlusion,
 author={Feng, Qi and Shum, Hubert P. H. and Morishima, Shigeo},
 booktitle={Proceedings of the 2018 Meeting on Image Recognition and Understanding},
 series={MIRU '18},
 title={Occlusion for 3D Object Manipulation with Hands in Augmented Reality},
 year={2018},
 month={8},
 numpages={4},
 location={Sapporo, Japan},
}

RIS

TY  - CONF
AU  - Feng, Qi
AU  - Shum, Hubert P. H.
AU  - Morishima, Shigeo
T2  - Proceedings of the 2018 Meeting on Image Recognition and Understanding
TI  - Occlusion for 3D Object Manipulation with Hands in Augmented Reality
PY  - 2018
Y1  - 8 2018
ER  - 

Plain Text

Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Occlusion for 3D Object Manipulation with Hands in Augmented Reality," in MIRU '18: Proceedings of the 2018 Meeting on Image Recognition and Understanding, Sapporo, Japan, Aug 2018.

Supporting Grants

Similar Research

Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Resolving Occlusion for 3D Object Manipulation with Hands in Mixed Reality", Proceedings of the 2018 ACM Symposium on Virtual Reality Software and Technology (VRST) Posters, 2018
Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Resolving Hand-Object Occlusion for Mixed Reality with Joint Deep Learning and Model Optimization", Computer Animation and Virtual Worlds (CAVW) - Proceedings of the 2020 International Conference on Computer Animation and Social Agents (CASA), 2020
Kanglei Zhou, Chen Chen, Yue Ma, Zhiying Leng, Hubert P. H. Shum, Frederick W. B. Li and Xiaohui Liang, "A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments", Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2023
Kanglei Zhou, Jiaying Chen, Hubert P. H. Shum, Frederick W. B. Li and Xiaohui Liang, "STGAE: Spatial Temporal Graph Auto-Encoder for Hand Motion Denoising", Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2021

 

 

Last updated on 25 March 2024
RSS Feed