[Recruitment]
Associate/Assistant Professor in Artificial Intelligence for Space-Enabled Technologies, Durham University

We are looking for applicants in Artificial Intelligence, Computer Vision, Edge Computing, Digital Twins, Human Computer Interaction, User Modelling, Robotics or Resilient Computing with potentials/achievements in informing space applications.

The post hoder will enjoy 1) a permanent (equivalent to US tenured) position at a top 100 university, 2) significantly reduced teaching, 3) a fully-funded PhD, 4) travel budget, 5) chance for a 2-year fully-funded Post-Doc.

Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation

Qi Feng, Hubert P. H. Shum and Shigeo Morishima
Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2023

Core A* Conference Core A* Conference

Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation
‡ According to Core Ranking 2023"

Abstract

Pre-captured immersive environments using omnidirectional cameras provide a wide range of virtual reality applications. Previous research has shown that manipulating the eye height in egocentric virtual environments can significantly affect distance perception and immersion. However, the influence of eye height in pre-captured real environments has received less attention due to the difficulty of altering the perspective after finishing the capture process. To explore this influence, we first propose a pilot study that captures real environments with multiple eye heights and asks participants to judge the egocentric distances and immersion. If a significant influence is confirmed, an effective image-based approach to adapt pre-captured real-world environments to the user’s eye height would be desirable. Motivated by the study, we propose a learning-based approach for synthesizing novel views for omnidirectional images with altered eye heights. This approach employs a multitask architecture that learns depth and semantic segmentation in two formats, and generates high-quality depth and semantic segmentation to facilitate the inpainting stage. With the improved omnidirectional-aware layered depth image, our approach synthesizes natural and realistic visuals for eye height adaptation. Quantitative and qualitative evaluation shows favorable results against state-of-the-art methods, and an extensive user study verifies improved perception and immersion for pre-captured real-world environments.

Downloads

YouTube

Citations

BibTeX

@inproceedings{feng23enhancing,
 author={Feng, Qi and Shum, Hubert P. H. and Morishima, Shigeo},
 booktitle={Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality},
 series={ISMAR '23},
 title={Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation},
 year={2023},
 month={10},
 pages={405-414},
 numpages={10},
 doi={10.1109/ISMAR59233.2023.00055},
 publisher={IEEE},
 location={Sydney, Australia},
}

RIS

TY  - CONF
AU  - Feng, Qi
AU  - Shum, Hubert P. H.
AU  - Morishima, Shigeo
T2  - Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality
TI  - Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation
PY  - 2023
Y1  - 10 2023
SP  - 405-414
EP  - 405-414
DO  - 10.1109/ISMAR59233.2023.00055
PB  - IEEE
ER  - 

Plain Text

Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation," in ISMAR '23: Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality, pp. 405-414, Sydney, Australia, IEEE, Oct 2023.

Supporting Grants

Similar Research

Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "360 Depth Estimation in the Wild - The Depth360 Dataset and the SegFuse Network", Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2022
Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Foreground-Aware Dense Depth Estimation for 360 Images", Journal of WSCG - Proceedings of the 2020 International Conferences in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), 2020
Qi Feng, Hubert P. H. Shum and Shigeo Morishima, "Bi-Projection Based Foreground-Aware Omnidirectional Depth Prediction", Proceedings of the 2021 Visual Computing (VC), 2021

 

 

Last updated on 17 February 2024
RSS Feed