Self-funded PhD Positions Available

Deep Learning based Computer Graphics for Creating Virtual Characters
Computer Vision with Deep Learning for Human Data Modelling
Computer Vision for Drones/UAVs Tracking from Video
Analyzing the Humans Context in Artworks and Cartoon with Computer Vision
Machine Learning based Human Data Analysis for Automatic Diseases Diagnosis

Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning

Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning

Abstract

Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketch image. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deep neural network that takes a 2D sketch and generates a set of multi-view depth and mask images, which form a more effective representation comparing to 3D meshes, and can be effectively fused to generate a 3D car shape. Since global models like deep learning have limited capacity to reconstruct fine-detail features, we propose a local lazy learning approach that constructs a small subspace based on a few relevant car samples in the database. Due to the small size of such a subspace, fine details can be represented effectively with a small number of parameters. With a low-cost optimization process, a high-quality car shape with detailed features is created. Experimental results show that the system performs consistently to create highly realistic cars of substantially different shape and topology.

Publication

Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima,
"Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning",
Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications (GRAPP)
, 2020
Citation: 1## Best Student Paper Award

## Citation counts from Google Scholar as of 2021

Downloads

Thumbnail
Paper (1.9MB)
Thumbnail
Video (258.2MB)
Thumbnail
Presentation (172.5MB)
Thumbnail
DOI - Publisher's Page

YouTube

References

BibTeX

@inproceedings{nozawa20single,
 author={Nozawa, Naoki and Shum, Hubert P. H. and Ho, Edmond S. L. and Morishima, Shigeo},
 booktitle={Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications},
 series={GRAPP '20},
 title={Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning},
 year={2020},
 month={Feb},
 pages={179--190},
 numpages={12},
 doi={10.5220/0009157001790190},
 issn={2184-4321},
 isbn={978-989-758-402-2},
 publisher={SciTePress},
 location={Valletta, Malta},
}

EndNote/RefMan

TY  - CONF
AU  - Nozawa, Naoki
AU  - Shum, Hubert P. H.
AU  - Ho, Edmond S. L.
AU  - Morishima, Shigeo
T2  - Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications
TI  - Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning
PY  - 2020
Y1  - Feb 2020
SP  - 179
EP  - 190
DO  - 10.5220/0009157001790190
SN  - 2184-4321
PB  - SciTePress
ER  - 

Plain Text

Naoki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho and Shigeo Morishima, "Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning," in GRAPP '20: Proceedings of the 2020 International Conference on Computer Graphics Theory and Applications, pp. 179-190, Valletta, Malta, SciTePress, Feb 2020.

Similar Research

 

Last update: 16 June 2021