Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars

Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars

Abstract

The control of self-driving cars has received growing attention recently. While existing research shows promising results in vehicle control using video from a monocular dash camera, there has been very limited work on directly learning vehicle control from motion-based cues. Such cues are powerful features for visual representations, as they encode the per-pixel movement between two consecutive images, allowing a system to effectively map the features into the control signal. We propose a new framework that exploits the use of a motion-based feature known as optical flow extracted from the dash camera, and demonstrates that such a feature is effective in significantly improving the accuracy of the control signals. Our proposed framework involves two main components. The flow predictor, as a self-supervised deep network, models the underlying scene structure from consecutive frames and generates the optical flow. The controller, as a supervised multi-task deep network, predicts both steer angle and speed. We demonstrate that the proposed framework using the optical flow features can effectively predict control signals from a dash camera video. Using the Cityscapes dataset, we validate that the system prediction has errors as low as 0.0130 rad/s on steer angle and 0.0615 m/s on speed, outperforming existing research.

Publication

Yuan Hu, Hubert P. H. Shum and Edmond S. L. Ho,
"Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars",
IET Intelligent Transport Systems (ITS)
, 2020
Impact Factor: 2.496#

# Impact factors from the Journal Citation Reports 2020

Downloads

Thumbnail Thumbnail Thumbnail Thumbnail Thumbnail Thumbnail Thumbnail Thumbnail
Paper (2.1MB)
Thumbnail
DOI - Publisher's Page

YouTube

References

BibTeX

@article{hu21multitask,
 author={Hu, Yuan and Shum, Hubert P. H. and Ho, Edmond S. L.},
 journal={IET Intelligent Transport Systems},
 series={ITS '21},
 title={Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars},
 year={2020},
 volume={14},
 number={13},
 pages={1845--1854},
 numpages={10},
 doi={10.1049/iet-its.2020.0439},
 issn={1751-956X},
 publisher={Institution of Engineering and Technology},
}

EndNote/RefMan

TY  - JOUR
AU  - Hu, Yuan
AU  - Shum, Hubert P. H.
AU  - Ho, Edmond S. L.
T2  - IET Intelligent Transport Systems
TI  - Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars
PY  - 2020
VL  - 14
IS  - 13
SP  - 1845
EP  - 1854
DO  - 10.1049/iet-its.2020.0439
SN  - 1751-956X
PB  - Institution of Engineering and Technology
ER  - 

Plain Text

Yuan Hu, Hubert P. H. Shum and Edmond S. L. Ho, "Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars," IET Intelligent Transport Systems, vol. 14, no. 13, pp. 1845-1854, Institution of Engineering and Technology, 2020.

Similar Research

 

 
 

Last updated on 02 December 2021, RSS Feeds