[Recruitment]
Associate/Assistant Professor in Artificial Intelligence for Space-Enabled Technologies, Durham University

We are looking for applicants in Artificial Intelligence, Computer Vision, Edge Computing, Digital Twins, Human Computer Interaction, User Modelling, Robotics or Resilient Computing with potentials/achievements in informing space applications.

The post hoder will enjoy 1) a permanent (equivalent to US tenured) position at a top 100 university, 2) significantly reduced teaching, 3) a fully-funded PhD, 4) travel budget, 5) chance for a 2-year fully-funded Post-Doc.

Automatic Sign Dance Synthesis from Gesture-Based Sign Language

Naoya Iwamoto, Hubert P. H. Shum, Wakana Asahina and Shigeo Morishima
Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG), 2019

Automatic Sign Dance Synthesis from Gesture-Based Sign Language

Abstract

Automatic dance synthesis has become more and more popular due to the increasing demand in computer games and animations. Existing research generates dance motions without much consideration for the context of the music. In reality, professional dancers make choreography according to the lyrics and music features. In this research, we focus on a particular genre of dance known as sign dance, which combines gesture-based sign language with full body dance motion. We propose a system to automatically generate sign dance from a piece of music and its corresponding sign gesture. The core of the system is a Sign Dance Model trained by multiple regression analysis to represent the correlations between sign dance and sign gesture/music, as well as a set of objective functions to evaluate the quality of the sign dance. Our system can be applied to music visualization, allowing people with hearing difficulties to understand and enjoy music.

Downloads

YouTube

Citations

BibTeX

@inproceedings{iwamoto19automatic,
 author={Iwamoto, Naoya and Shum, Hubert P. H. and Asahina, Wakana and Morishima, Shigeo},
 booktitle={Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games},
 series={MIG '19},
 title={Automatic Sign Dance Synthesis from Gesture-Based Sign Language},
 year={2019},
 month={10},
 pages={18:1--18:9},
 numpages={9},
 doi={10.1145/3359566.3360069},
 isbn={978-1-4503-6994-7},
 publisher={ACM},
 Address={New York, NY, USA},
 location={Newcastle upon Tyne, UK},
}

RIS

TY  - CONF
AU  - Iwamoto, Naoya
AU  - Shum, Hubert P. H.
AU  - Asahina, Wakana
AU  - Morishima, Shigeo
T2  - Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games
TI  - Automatic Sign Dance Synthesis from Gesture-Based Sign Language
PY  - 2019
Y1  - 10 2019
SP  - 18:1
EP  - 18:9
DO  - 10.1145/3359566.3360069
SN  - 978-1-4503-6994-7
PB  - ACM
ER  - 

Plain Text

Naoya Iwamoto, Hubert P. H. Shum, Wakana Asahina and Shigeo Morishima, "Automatic Sign Dance Synthesis from Gesture-Based Sign Language," in MIG '19: Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games, pp. 18:1-18:9, Newcastle upon Tyne, UK, ACM, Oct 2019.

Supporting Grants

The Ministry of Education, Culture, Sports, Science and Technology

MEXT Top Global University Project Scholarship (Ref: ): £5,500, Co-Applicant (PI: Prof. Shigeo Morishima, Japanese Partner)
Received from The Ministry of Education, Culture, Sports, Science and Technology, 2015-2016
Project Page

Similar Research

Wakana Asahina, Naoya Iwamoto, Hubert P. H. Shum and Shigeo Morishima, "Automatic Dance Generation System Considering Sign Language Information", Proceedings of the 2016 ACM SIGGRAPH Posters, 2016
Naoya Iwamoto, Takuya Kato, Hubert P. H. Shum, Ryo Kakitsuka, Kenta Hara and Shigeo Morishima, "DanceDJ: A 3D Dance Animation Authoring System for Live Performance", Proceedings of the 2017 International Conference on Advances in Computer Entertainment Technology (ACE), 2017
Hubert P. H. Shum, Taku Komura and Pranjul Yadav, "Angular Momentum Guided Motion Concatenation", Computer Animation and Virtual Worlds (CAVW) - Proceedings of the 2009 International Conference on Computer Animation and Social Agents (CASA), 2009
Liuyang Zhou, Lifeng Shang, Hubert P. H. Shum and Howard Leung, "Human Motion Variation Synthesis with Multivariate Gaussian Processes", Computer Animation and Virtual Worlds (CAVW) - Proceedings of the 2014 International Conference on Computer Animation and Social Agents (CASA), 2014
Edmond S. L. Ho, Hubert P. H. Shum, He Wang and Li Yi, "Synthesizing Motion with Relative Emotion Strength", Proceedings of the 2017 ACM SIGGRAPH Asia Workshop on Data-Driven Animation Techniques (D2AT), 2017
Hubert P. H. Shum, Ludovic Hoyet, Edmond S. L. Ho, Taku Komura and Franck Multon, "Natural Preparation Behavior Synthesis", Computer Animation and Virtual Worlds (CAVW), 2013
He Wang, Edmond S. L. Ho, Hubert P. H. Shum and Zhanxing Zhu, "Spatio-Temporal Manifold Learning for Human Motions via Long-Horizon Modeling", IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021
Hubert P. H. Shum, Ludovic Hoyet, Edmond S. L. Ho, Taku Komura and Franck Multon, "Preparation Behaviour Synthesis with Reinforcement Learning", Proceedings of the 2013 International Conference on Computer Animation and Social Agents (CASA), 2013

 

 

Last updated on 17 February 2024
RSS Feed