A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments

Kanglei Zhou, Chen Chen, Yue Ma, Zhiying Leng, Hubert P. H. Shum, Frederick W. B. Li and Xiaohui Liang
Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2023

Core A* Conference Core A* Conference

A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments
‡ According to Core Ranking 2023"

Abstract

As human exploration of space continues to progress, the use of Mixed Reality (MR) for simulating microgravity environments and facilitating training in hand-object interaction holds immense practical significance. However, hand-object interaction in microgravity presents distinct challenges compared to terrestrial environments due to the absence of gravity. This results in heightened agility and inherent unpredictability of movements that traditional methods struggle to simulate accurately. To this end, we propose a novel MR-based hand-object interaction system in simulated microgravity environments, leveraging physics-based simulations to enhance the interaction between the user’s real hand and virtual objects. Specifically, we introduce a physics-based hand-object interaction model that combines impulse-based simulation with penetration contact dynamics. This accurately captures the intricacies of hand-object interaction in microgravity. By considering forces and impulses during contact, our model ensures realistic collision responses and enables effective object manipulation in the absence of gravity. The proposed system presents a cost-effective solution for users to simulate object manipulation in microgravity. It also holds promise for training space travelers, equipping them with greater immersion to better adapt to space missions. The system reliability and fidelity test verifies the superior effectiveness of our system compared to the state-of-the-art CLAP system.

Downloads

YouTube

Citations

BibTeX

@inproceedings{zhou23mixed,
 author={Zhou, Kanglei and Chen, Chen and Ma, Yue and Leng, Zhiying and Shum, Hubert P. H. and Li, Frederick W. B. and Liang, Xiaohui},
 booktitle={Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality},
 series={ISMAR '23},
 title={A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments},
 year={2023},
 month={10},
 pages={167--176},
 numpages={10},
 doi={10.1109/ISMAR59233.2023.00031},
 publisher={IEEE},
 location={Sydney, Australia},
}

RIS

TY  - CONF
AU  - Zhou, Kanglei
AU  - Chen, Chen
AU  - Ma, Yue
AU  - Leng, Zhiying
AU  - Shum, Hubert P. H.
AU  - Li, Frederick W. B.
AU  - Liang, Xiaohui
T2  - Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality
TI  - A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments
PY  - 2023
Y1  - 10 2023
SP  - 167
EP  - 176
DO  - 10.1109/ISMAR59233.2023.00031
PB  - IEEE
ER  - 

Plain Text

Kanglei Zhou, Chen Chen, Yue Ma, Zhiying Leng, Hubert P. H. Shum, Frederick W. B. Li and Xiaohui Liang, "A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments," in ISMAR '23: Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality, pp. 167-176, Sydney, Australia, IEEE, Oct 2023.

Supporting Grants

Similar Research

Kanglei Zhou, Jiaying Chen, Hubert P. H. Shum, Frederick W. B. Li and Xiaohui Liang, "STGAE: Spatial Temporal Graph Auto-Encoder for Hand Motion Denoising", Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2021
Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Resolving Hand-Object Occlusion for Mixed Reality with Joint Deep Learning and Model Optimization", Computer Animation and Virtual Worlds (CAVW) - Proceedings of the 2020 International Conference on Computer Animation and Social Agents (CASA), 2020
Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Resolving Occlusion for 3D Object Manipulation with Hands in Mixed Reality", Proceedings of the 2018 ACM Symposium on Virtual Reality Software and Technology (VRST) Posters, 2018
Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Occlusion for 3D Object Manipulation with Hands in Augmented Reality", Proceedings of the 2018 Meeting on Image Recognition and Understanding (MIRU), 2018

 

 

Last updated on 14 April 2024
RSS Feed