Self-funded PhD Positions Available

Biomedical Engineering with Deep Learning based Video Analysis
Computer Vision with Deep Learning for Human Data Modelling
Deep Learning based Computer Graphics for Creating Virtual Characters

Illumination-aware Multi-task GANs for Foreground Segmentation

Illumination-aware Multi-task GANs for Foreground Segmentation

Abstract

Foreground-background segmentation has been an active research area over the years. However, conventional models fail to produce accurate results when challenged with videos of challenging illumination conditions. In this paper, we present a robust model that allows accurately extracting the foreground even in exceptionally dark or bright scenes, as well as continuously varying illumination in a video sequence. This is accomplished by a triple multi-task generative adversarial network (TMT-GAN) that effectively models the semantic relationship between dark and bright images, and performs binary segmentation end-to-end. Our contribution is two-fold: First, we show that by jointly optimising the GAN loss and the segmentation loss, our network simultaneously learns both tasks that mutually benefit each other. Secondly, fusing features of images with varying illumination into the segmentation branch vastly improves the performance of the network. Comparative evaluations on highly challenging real and synthetic benchmark datasets (ESI, SABS) demonstrate the robustness of TMT-GAN and its superiority over state-of-the-art approaches.

Publication

Dimitrios Sakkos, Edmond S. L. Ho and Hubert P. H. Shum,
"Illumination-aware Multi-task GANs for Foreground Segmentation",
IEEE Access
, 2019
Impact Factor: 3.476# Citation: 18##

# Impact factors from the Journal Citation Reports 2021
## Citation counts from Google Scholar as of 2022

Downloads

YouTube

References

BibTeX

@article{sakkos19illuminationaware,
 author={Sakkos, Dimitrios and Ho, Edmond S. L. and Shum, Hubert P. H.},
 journal={IEEE Access},
 title={Illumination-aware Multi-task GANs for Foreground Segmentation},
 year={2019},
 volume={7},
 number={1},
 pages={10976--10986},
 numpages={12},
 doi={10.1109/ACCESS.2019.2891943},
 issn={2169-3536},
 publisher={IEEE},
}

RIS

TY  - JOUR
AU  - Sakkos, Dimitrios
AU  - Ho, Edmond S. L.
AU  - Shum, Hubert P. H.
T2  - IEEE Access
TI  - Illumination-aware Multi-task GANs for Foreground Segmentation
PY  - 2019
VL  - 7
IS  - 1
SP  - 10976
EP  - 10986
DO  - 10.1109/ACCESS.2019.2891943
SN  - 2169-3536
PB  - IEEE
ER  - 

Plain Text

Dimitrios Sakkos, Edmond S. L. Ho and Hubert P. H. Shum, "Illumination-aware Multi-task GANs for Foreground Segmentation," IEEE Access, vol. 7, no. 1, pp. 10976-10986, IEEE, 2019.

Similar Research

 

 
 

Last updated on 01 August 2022, RSS Feeds