Depth-Aware Endoscopic Video Inpainting

Francis Xiatian Zhang, Shuang Chen, Xianghua Xie and Hubert P. H. Shum
Proceedings of the 2024 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2024

Core A Conference H5-Index: 96# Core A Conference

Depth-Aware Endoscopic Video Inpainting
‡ According to Core Ranking 2023
# According to Google Scholar 2024

Abstract

Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos, they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework’s superiority, achieving a 2% improvement in PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting.

Downloads

YouTube

Citations

BibTeX

@inproceedings{zhang24endoscopic,
 author={Zhang, Francis Xiatian and Chen, Shuang and Xie, Xianghua and Shum, Hubert P. H.},
 booktitle={Proceedings of the 2024 International Conference on Medical Image Computing and Computer Assisted Intervention},
 series={MICCAI '24},
 title={Depth-Aware Endoscopic Video Inpainting},
 year={2024},
 publisher={Springer},
 location={Marrakesh, Morocco},
}

RIS

TY  - CONF
AU  - Zhang, Francis Xiatian
AU  - Chen, Shuang
AU  - Xie, Xianghua
AU  - Shum, Hubert P. H.
T2  - Proceedings of the 2024 International Conference on Medical Image Computing and Computer Assisted Intervention
TI  - Depth-Aware Endoscopic Video Inpainting
PY  - 2024
PB  - Springer
ER  - 

Plain Text

Francis Xiatian Zhang, Shuang Chen, Xianghua Xie and Hubert P. H. Shum, "Depth-Aware Endoscopic Video Inpainting," in MICCAI '24: Proceedings of the 2024 International Conference on Medical Image Computing and Computer Assisted Intervention, Marrakesh, Morocco, Springer, 2024.

Supporting Grants

Similar Research

Francis Xiatian Zhang, Noura Al Moubayed and Hubert P. H. Shum, "Towards Graph Representation Learning Based Surgical Workflow Anticipation", Proceedings of the 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), 2022
Haozheng Zhang, Edmond S. L. Ho, Francis Xiatian Zhang and Hubert P. H. Shum, "Pose-Based Tremor Classification for Parkinson’s Disease Diagnosis from Video", Proceedings of the 2022 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2022
Kanglei Zhou, Ruizhi Cai, Yue Ma, Qingqing Tan, Xinning Wang, Jianguo Li, Hubert P. H. Shum, Frederick W. B. Li, Song Jin and Xiaohui Liang, "A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis", IEEE Transactions on Visualization and Computer Graphics (TVCG) - Proceedings of the 2023 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2023

 

 

Last updated on 15 July 2024
RSS Feed