A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching

Shanfeng Hu, Hubert P. H. Shum, Nauman Aslam, Frederick W. B. Li and Xiaohui Liang
IEEE Transactions on Multimedia (TMM), 2020

REF 2021 Submitted Output Impact Factor: 8.4 Top 10% Journal in Computer Science, Software Engineering

A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching

Abstract

In this paper, we propose a deep metric for unifying the representation of mesh saliency detection and non-rigid shape matching. While saliency detection and shape matching are two closely related and fundamental tasks in shape analysis, previous methods approach them separately and independently, failing to exploit their mutually beneficial underlying relationship. In view of the existing gap between saliency and matching, we propose to solve them together using a unified metric representation of surface meshes. We show that saliency and matching can be rigorously derived from our representation as the principal eigenvector and the smoothed Laplacian eigenvectors respectively. Learning the representation jointly allows matching to improve the deformation-invariance of saliency while allowing saliency to improve the feature localization of matching. To parameterize the representation from a mesh, we also propose a deep recurrent neural network (RNN) for effectively integrating multi-scale shape features and a soft-thresholding operator for adaptively enhancing the sparsity of saliency. Results show that by jointly learning from a pair of saliency and matching datasets, matching improves the accuracy of detected salient regions on meshes, which is especially obvious for small-scale saliency datasets, such as those having one to two meshes. At the same time, saliency improves the accuracy of shape matchings among meshes with reduced matching errors on surfaces.


Downloads


YouTube


Cite This Research

Plain Text

Shanfeng Hu, Hubert P. H. Shum, Nauman Aslam, Frederick W. B. Li and Xiaohui Liang, "A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching," IEEE Transactions on Multimedia, vol. 22, no. 9, pp. 2278-2292, IEEE, 2020.

BibTeX

@article{hu20deep,
 author={Hu, Shanfeng and Shum, Hubert P. H. and Aslam, Nauman and Li, Frederick W. B. and Liang, Xiaohui},
 journal={IEEE Transactions on Multimedia},
 series={TMM '24},
 title={A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching},
 year={2020},
 volume={22},
 number={9},
 pages={2278--2292},
 numpages={15},
 doi={10.1109/TMM.2019.2952983},
 issn={1941-0077},
 publisher={IEEE},
}

RIS

TY  - JOUR
AU  - Hu, Shanfeng
AU  - Shum, Hubert P. H.
AU  - Aslam, Nauman
AU  - Li, Frederick W. B.
AU  - Liang, Xiaohui
T2  - IEEE Transactions on Multimedia
TI  - A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching
PY  - 2020
VL  - 22
IS  - 9
SP  - 2278
EP  - 2292
DO  - 10.1109/TMM.2019.2952983
SN  - 1941-0077
PB  - IEEE
ER  - 


Supporting Grants

The Royal Society
Modelling Human Motion for Synthesis and Recognition with Deep Learning on Surface Features
Royal Society International Exchanges (Ref: IES\R2\181024): £12,000, Principal Investigator ()
Received from The Royal Society, UK, 2019-2022
Project Page
Erasmus Mundus
Sustainable Green Economies through Learning, Innovation, Networking and Knowledge Exchange (gLink)
Erasmus Mundus Action 2 Programme (Ref: 2014-0861/001-001): €3.03 million, Northumbria University Funding Management Leader (PI: Prof. Nauman Aslam)
Received from Erasmus Mundus, 2015-2018
Project Page

Similar Research

Shanfeng Hu, Xiaohui Liang, Hubert P. H. Shum, Frederick W. B. Li and Nauman Aslam, "Sparse Metric-Based Mesh Saliency", Neurocomputing, 2020
Shanfeng Hu, Hubert P. H. Shum and Antonio Mucherino, "DSPP: Deep Shape and Pose Priors of Humans", Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG), 2019
Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "3D Car Shape Reconstruction from a Single Sketch Image", Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) Posters, 2019
Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "Single Sketch Image Based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning", Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications (GRAPP), 2020
Ziyi Chang, George Alex Koulieris and Hubert P. H. Shum, "3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models", Proceedings of the 2022 ACM Symposium on Virtual Reality Software and Technology (VRST), 2022

 

Last updated on 6 October 2024
RSS Feed