Jiajun Deng, Shaoshuai Shi, Peiwei Li, Wengang Zhou, Yanyong Zhang, Houqiang Li
Recent advances on 3D object detection heavily rely on how the 3D data are represented, \emph{i.e.}, voxel-based or point-based representation. Many existing high performance 3D detectors are point-based because this structure can better retain precise point positions. Nevertheless, point-level features lead to high computation overheads due to unordered storage. In contrast, the voxel-based structure is better suited for feature extraction but often yields lower accuracy because the input data are divided into grids. In this paper, we take a slightly different viewpoint -- we find that precise positioning of raw points is not essential for high performance 3D object detection and that the coarse voxel granularity can also offer sufficient detection accuracy. Bearing this view in mind, we devise a simple but effective voxel-based framework, named Voxel R-CNN. By taking full advantage of voxel features in a two stage approach, our method achieves comparable detection accuracy with state-of-the-art point-based models, but at a fraction of the computation cost. Voxel R-CNN consists of a 3D backbone network, a 2D bird-eye-view (BEV) Region Proposal Network and a detect head. A voxel RoI pooling is devised to extract RoI features directly from voxel features for further refinement. Extensive experiments are conducted on the widely used KITTI Dataset and the more recent Waymo Open Dataset. Our results show that compared to existing voxel-based methods, Voxel R-CNN delivers a higher detection accuracy while maintaining a real-time frame processing rate, \emph{i.e}., at a speed of 25 FPS on an NVIDIA RTX 2080 Ti GPU. The code is available at \url{https://github.com/djiajunustc/Voxel-R-CNN}.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Object Detection | KITTI Cars Hard val | AP | 78.93 | Voxel R-CNN |
| Object Detection | KITTI Cars Hard | AP | 77.06 | Voxel R-CNN |
| Object Detection | KITTI Cars Moderate val | AP | 84.52 | Voxel R-CNN |
| Object Detection | KITTI Cars Easy val | AP | 89.41 | Voxel R-CNN |
| 3D | KITTI Cars Hard val | AP | 78.93 | Voxel R-CNN |
| 3D | KITTI Cars Hard | AP | 77.06 | Voxel R-CNN |
| 3D | KITTI Cars Moderate val | AP | 84.52 | Voxel R-CNN |
| 3D | KITTI Cars Easy val | AP | 89.41 | Voxel R-CNN |
| 3D Object Detection | KITTI Cars Hard val | AP | 78.93 | Voxel R-CNN |
| 3D Object Detection | KITTI Cars Hard | AP | 77.06 | Voxel R-CNN |
| 3D Object Detection | KITTI Cars Moderate val | AP | 84.52 | Voxel R-CNN |
| 3D Object Detection | KITTI Cars Easy val | AP | 89.41 | Voxel R-CNN |
| 2D Classification | KITTI Cars Hard val | AP | 78.93 | Voxel R-CNN |
| 2D Classification | KITTI Cars Hard | AP | 77.06 | Voxel R-CNN |
| 2D Classification | KITTI Cars Moderate val | AP | 84.52 | Voxel R-CNN |
| 2D Classification | KITTI Cars Easy val | AP | 89.41 | Voxel R-CNN |
| 2D Object Detection | KITTI Cars Hard val | AP | 78.93 | Voxel R-CNN |
| 2D Object Detection | KITTI Cars Hard | AP | 77.06 | Voxel R-CNN |
| 2D Object Detection | KITTI Cars Moderate val | AP | 84.52 | Voxel R-CNN |
| 2D Object Detection | KITTI Cars Easy val | AP | 89.41 | Voxel R-CNN |
| 16k | KITTI Cars Hard val | AP | 78.93 | Voxel R-CNN |
| 16k | KITTI Cars Hard | AP | 77.06 | Voxel R-CNN |
| 16k | KITTI Cars Moderate val | AP | 84.52 | Voxel R-CNN |
| 16k | KITTI Cars Easy val | AP | 89.41 | Voxel R-CNN |