3D Car Shape Reconstruction from a Single Sketch Image

Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima
Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) Posters, 2019

 Best Poster Award

3D Car Shape Reconstruction from a Single Sketch Image

Abstract

Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketch image. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deep neural network that takes a 2D sketch and generates a set of multiview depth & mask images, which are more effective representation comparing to 3D mesh, and can be combined to form the 3D car shape. To ensure the volume and diversity of the training data, we propose a feature-preserving car mesh augmentation pipeline for data augmentation. Since deep learning has limited capacity to reconstruct fine-detail features, we propose a lazy learning approach that constructs a small subspace based on a few relevant car samples in the database. Due to the small size of such a subspace, fine details can be represented effectively with a small number of parameters. With a low-cost optimization process, a high-quality car with detailed features is created. Experimental results show that the system performs consistently to create highly realistic cars of substantially different shape and topology, with a very low computational cost.

Downloads

YouTube

Citations

BibTeX

@inproceedings{nozawa193dcar,
 author={Nozawa, Naoki and Shum, Hubert P. H. and Ho, Edmond S. L. and Morishima, Shigeo},
 booktitle={Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games},
 series={MIG '19},
 title={3D Car Shape Reconstruction from a Single Sketch Image},
 year={2019},
 month={10},
 pages={37:1--37:2},
 numpages={2},
 doi={10.1145/3359566.3364693},
 isbn={978-1-4503-6994-7},
 publisher={ACM},
 Address={New York, NY, USA},
 location={Newcastle upon Tyne, UK},
}

RIS

TY  - CONF
AU  - Nozawa, Naoki
AU  - Shum, Hubert P. H.
AU  - Ho, Edmond S. L.
AU  - Morishima, Shigeo
T2  - Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games
TI  - 3D Car Shape Reconstruction from a Single Sketch Image
PY  - 2019
Y1  - 10 2019
SP  - 37:1
EP  - 37:2
DO  - 10.1145/3359566.3364693
SN  - 978-1-4503-6994-7
PB  - ACM
ER  - 

Plain Text

Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "3D Car Shape Reconstruction from a Single Sketch Image," in MIG '19: Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games, pp. 37:1-37:2, Newcastle upon Tyne, UK, ACM, Oct 2019.

Supporting Grants

Waseda University

Graduate Program for Embodiment Informatics (Ref: H29-LDGSN-024 & H29-LDGSN-025): ¥2.93 million (~£20,000), Co-Applicant (PI: Prof. Shigeo Morishima, Japanese Partner)
Received from Waseda University, Japan, 2017-2018
Project Page

Similar Research

Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "Single Sketch Image Based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning", Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications (GRAPP), 2020
Naoki Nozawa, Hubert P. H. Shum, Qi Feng, Edmond S. L. Ho and Shigeo Morishima, "3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning", Visual Computer (VC), 2022
Ziyi Chang, George Alex Koulieris and Hubert P. H. Shum, "3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models", Proceedings of the 2022 ACM Symposium on Virtual Reality Software and Technology (VRST), 2022
Shanfeng Hu, Hubert P. H. Shum, Nauman Aslam, Frederick W. B. Li and Xiaohui Liang, "A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching", IEEE Transactions on Multimedia (TMM), 2020
Jingtian Zhang, Hubert P. H. Shum, Kevin D. McCay and Edmond S. L. Ho, "Prior-Less 3D Human Shape Reconstruction with an Earth Mover's Distance Informed CNN", Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) Posters, 2019

 

 

Last updated on 14 April 2024
RSS Feed