MxT: Mamba x Transformer for Image Inpainting

Shuang Chen, Amir Atapour-Abarghouei, Haozheng Zhang and Hubert P. H. Shum
Proceedings of the 2024 British Machine Vision Conference (BMVC), 2024

 H5-Index: 65#

MxT: Mamba x Transformer for Image Inpainting
# According to Google Scholar 2024

Abstract

Image inpainting, or image completion, is a crucial task in computer vision that aims to restore missing or damaged regions of images with semantically coherent content. This technique requires a precise balance of local texture replication and global contextual understanding to ensure the restored image integrates seamlessly with its surroundings. Traditional methods using Convolutional Neural Networks (CNNs) are effective at capturing local patterns but often struggle with broader contextual relationships due to the limited receptive fields. Recent advancements have incorporated transformers, leveraging their ability to understand global interactions. However, these methods face computational inefficiencies and struggle to maintain fine-grained details. To overcome these challenges, we introduce M×T composed of the proposed Hybrid Module (HM), which combines Mamba with the transformer in a synergistic manner. Mamba is adept at efficiently processing long sequences with linear computational costs, making it an ideal complement to the transformer for handling long-scale data interactions. Our HM facilitates dual-level interaction learning at both pixel and patch levels, greatly enhancing the model to reconstruct images with high quality and contextual accuracy. We evaluate M×T on the widely-used CelebA-HQ and Places2-standard datasets, where it consistently outperformed existing state-of-the-art methods.


Downloads


YouTube


Cite This Research

Plain Text

Shuang Chen, Amir Atapour-Abarghouei, Haozheng Zhang and Hubert P. H. Shum, "MxT: Mamba x Transformer for Image Inpainting," in BMVC '24: Proceedings of the 2024 British Machine Vision Conference, Glasgow, UK, 2024.

BibTeX

@inproceedings{chen24mxt,
 author={Chen, Shuang and Atapour-Abarghouei, Amir and Zhang, Haozheng and Shum, Hubert P. H.},
 booktitle={Proceedings of the 2024 British Machine Vision Conference},
 series={BMVC '24},
 title={MxT: Mamba x Transformer for Image Inpainting},
 year={2024},
 location={Glasgow, UK},
}

RIS

TY  - CONF
AU  - Chen, Shuang
AU  - Atapour-Abarghouei, Amir
AU  - Zhang, Haozheng
AU  - Shum, Hubert P. H.
T2  - Proceedings of the 2024 British Machine Vision Conference
TI  - MxT: Mamba x Transformer for Image Inpainting
PY  - 2024
ER  - 


Supporting Grants


Similar Research

Shuang Chen, Amir Atapour-Abarghouei and Hubert P. H. Shum, "HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention", IEEE Transactions on Multimedia (TMM), 2024

 

Last updated on 7 September 2024
RSS Feed