FDFlowNet: Fast Optical Flow Estimation using a Deep Lightweight Network

Abstract

Significant progress has been made for estimating optical flow using deep neural networks. Advanced deep models achieve accurate flow estimation often with a considerable computation complexity and time-consuming training processes. In this work, we present a lightweight yet effective model for real-time optical flow estimation, termed FDFlowNet (fast deep flownet). We achieve better or similar accuracy on the challenging KITTI and Sintel benchmarks while being about 2 times faster than PWC-Net. This is achieved by a carefully-designed structure and newly proposed components. We first introduce an U-shape network for constructing multi-scale feature which benefits upper levels with global receptive field compared with pyramid network. In each scale, a partial fully connected structure with dilated convolution is proposed for flow estimation that obtains a good balance among speed, accuracy and number of parameters compared with sequential connected and dense connected structures. Experiments demonstrate that our model achieves state-of-the-art performance while being fast and lightweight.