Facial Reshaping Operator for Controllable Face Beautification

Shanfeng Hu, Hubert P. H. Shum, Xiaohui Liang, Frederick W. B. Li and Nauman Aslam
Expert Systems with Applications (ESWA), 2021

 Impact Factor: 8.5 Top 25% Journal in Computer Science, Artificial Intelligence

Facial Reshaping Operator for Controllable Face Beautification


Posting attractive facial photos is part of everyday life in the social media era. Motivated by the demand, we propose a lightweight method to automatically and efficiently beautify the shapes of both portrait and non-portrait faces in photos, while allowing users to customize the beautification of individual facial features. Previous methods focus on the beautification of mostly frontal and neutral faces, without incorporating user controllability in the beautification process. To address these restrictions, we propose the Facial Reshaping Operator representation, which is affine-invariant, captures the pairwise geometric configuration of facial landmarks, and allows for efficient face beautification with the user-specified weights of individual facial parts. We also propose an unsupervised beautification method in the operator space of faces, where an input face is iteratively pulled towards a local nearby density mode with improved attractiveness. Our method distinguishes itself from the commercial beautification tools in that it mildly enhances facial shapes without altering makeups or complexions, which complements these tools that lack fine-grained control on the attractiveness of facial shapes for users. The experimental results show that our method improves facial shape attractiveness for a large range of poses and expressions, demonstrating the potential of applicability to photos seen on the social media such as Facebook and Instagram everyday.





 author={Hu, Shanfeng and Shum, Hubert P. H. and Liang, Xiaohui and Li, Frederick W. B. and Aslam, Nauman},
 journal={Expert Systems with Applications},
 title={Facial Reshaping Operator for Controllable Face Beautification},


AU  - Hu, Shanfeng
AU  - Shum, Hubert P. H.
AU  - Liang, Xiaohui
AU  - Li, Frederick W. B.
AU  - Aslam, Nauman
T2  - Expert Systems with Applications
TI  - Facial Reshaping Operator for Controllable Face Beautification
PY  - 2021
VL  - 167
SP  - 114067
EP  - 114067
DO  - 10.1016/j.eswa.2020.114067
SN  - 0957-4174
PB  - Elsevier
ER  - 

Plain Text

Shanfeng Hu, Hubert P. H. Shum, Xiaohui Liang, Frederick W. B. Li and Nauman Aslam, "Facial Reshaping Operator for Controllable Face Beautification," Expert Systems with Applications, vol. 167, pp. 114067, Elsevier, 2021.

Supporting Grants

Erasmus Mundus
Sustainable Green Economies through Learning, Innovation, Networking and Knowledge Exchange (gLink)
Erasmus Mundus Action 2 Programme (Ref: 2014-0861/001-001): €3.03 million, Co-Investigator, Northumbria University Funding Management Leader (PI: Prof. Nauman Aslam)
Received from Erasmus Mundus, 2015-2018
Project Page

Similar Research

Daniel Organisciak, Edmond S. L. Ho and Hubert P. H. Shum, "Makeup Style Transfer on Low-Quality Images with Weighted Multi-Scale Attention", Proceedings of the 2020 International Conference on Pattern Recognition (ICPR), 2020
Andreea Stef, Kaveen Perera, Hubert P. H. Shum and Edmond S. L. Ho, "Synthesizing Expressive Facial and Speech Animation by Text-to-IPA Translation with Emotion Control", Proceedings of the 2018 International Conference on Software, Knowledge, Information Management and Applications (SKIMA), 2018
Edmond S. L. Ho, Kevin D. McCay, Hubert P. H. Shum, Longzhi Yang, David Sainsbury and Peter Hodgkinson, "Patient Assessment Assistant Using Augmented Reality", Proceedings of the 2018 UK-China Newton Fund Researcher Links Workshop Health and Well-being Through VR and AR, 2018



Last updated on 4 June 2024
RSS Feed