3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning

Naoki Nozawa, Hubert P. H. Shum, Qi Feng, Edmond S. L. Ho and Shigeo Morishima
Visual Computer (VC), 2022

 Impact Factor: 3.5 Citation: 26#

3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning
# According to Google Scholar 2023"

Abstract

3D car models are heavily used in computer games, visual effects, and even automotive designs. As a result, producing such models with minimal labour costs is increasingly more important. To tackle the challenge, we propose a novel system to reconstruct a 3D car using a single sketch image. The system learns from a synthetic database of 3D car models and their corresponding 2D contour sketches and segmentation masks, allowing effective training with minimal data collection cost. The core of the system is a machine learning pipeline that combines the use of a Generative Adversarial Network (GAN) and lazy learning. GAN, being a deep learning method, is capable of modelling complicated data distributions, enabling the effective modelling of a large variety of cars. Its major weakness is that as a global method, modelling the fine details in the local region is challenging. Lazy learning works well to preserve local features by generating a local subspace with relevant data samples. We demonstrate that the combined use of GAN and lazy learning produces is able to produce high-quality results, in which different types of cars with complicated local features can be generated effectively with a single sketch. Our method outperforms existing ones using other machine learning structures such as the variational autoencoder.

Downloads

YouTube

Citations

BibTeX

@article{nozawa21car,
 author={Nozawa, Naoki and Shum, Hubert P. H. and Feng, Qi and Ho, Edmond S. L. and Morishima, Shigeo},
 journal={Visual Computer},
 title={3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning},
 year={2022},
 volume={38},
 number={4},
 pages={1317--1330},
 numpages={14},
 doi={10.1007/s00371-020-02024-y},
 issn={1432-2315},
 publisher={Springer},
}

RIS

TY  - JOUR
AU  - Nozawa, Naoki
AU  - Shum, Hubert P. H.
AU  - Feng, Qi
AU  - Ho, Edmond S. L.
AU  - Morishima, Shigeo
T2  - Visual Computer
TI  - 3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning
PY  - 2022
VL  - 38
IS  - 4
SP  - 1317
EP  - 1330
DO  - 10.1007/s00371-020-02024-y
SN  - 1432-2315
PB  - Springer
ER  - 

Plain Text

Naoki Nozawa, Hubert P. H. Shum, Qi Feng, Edmond S. L. Ho and Shigeo Morishima, "3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning," Visual Computer, vol. 38, no. 4, pp. 1317-1330, Springer, 2022.

Supporting Grants

Waseda University

Graduate Program for Embodiment Informatics (Ref: H29-LDGSN-024 & H29-LDGSN-025): ¥2.93 million (~£20,000), Co-Applicant (PI: Prof. Shigeo Morishima, Japanese Partner)
Received from Waseda University, Japan, 2017-2018
Project Page

Similar Research

Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "Single Sketch Image Based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning", Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications (GRAPP), 2020
Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "3D Car Shape Reconstruction from a Single Sketch Image", Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) Posters, 2019
Ziyi Chang, George Alex Koulieris and Hubert P. H. Shum, "3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models", Proceedings of the 2022 ACM Symposium on Virtual Reality Software and Technology (VRST), 2022
Jingtian Zhang, Hubert P. H. Shum, Kevin D. McCay and Edmond S. L. Ho, "Prior-Less 3D Human Shape Reconstruction with an Earth Mover's Distance Informed CNN", Proceedings of the 2019 ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) Posters, 2019

 

 

Last updated on 14 April 2024
RSS Feed