The superior performance of Deformable Convolutional Networks arises from its ability to adapt to the geometric variations of objects. Through an examination of its adaptive behavior, we observe that while the spatial support for its neural features conforms more closely than regular ConvNets to object structure, this support may nevertheless extend well beyond the region of interest, causing features to be influenced by irrelevant image content. To address this problem, we present a reformulation of Deformable ConvNets that improves its ability to focus on pertinent image regions, through increased modeling power and stronger training. The modeling power is enhanced through a more comprehensive integration of deformable convolution within the network, and by introducing a modulation mechanism that expands the scope of deformation modeling. To effectively harness this enriched modeling capability, we guide network training via a proposed feature mimicking scheme that helps the network to learn features that reflect the object focus and classification power of R-CNN features. With the proposed contributions, this new version of Deformable ConvNets yields significant performance gains over the original model and produces leading results on the COCO benchmark for object detection and instance segmentation.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Object Detection | COCO test-dev | AP50 | 67.9 | DCNv2 (ResNet-101, multi-scale) |
| Object Detection | COCO test-dev | AP75 | 50.8 | DCNv2 (ResNet-101, multi-scale) |
| Object Detection | COCO test-dev | APL | 59.5 | DCNv2 (ResNet-101, multi-scale) |
| Object Detection | COCO test-dev | APM | 49.1 | DCNv2 (ResNet-101, multi-scale) |
| Object Detection | COCO test-dev | APS | 27.8 | DCNv2 (ResNet-101, multi-scale) |
| Object Detection | COCO test-dev | box mAP | 46 | DCNv2 (ResNet-101, multi-scale) |
| Object Detection | COCO minival | box AP | 43.1 | Mask R-CNN (ResNet-101, DCNv2) |
| Object Detection | COCO minival | APL | 58.7 | Faster R-CNN (ResNet-101, DCNv2) |
| Object Detection | COCO minival | APM | 45.8 | Faster R-CNN (ResNet-101, DCNv2) |
| Object Detection | COCO minival | APS | 22.2 | Faster R-CNN (ResNet-101, DCNv2) |
| Object Detection | COCO minival | box AP | 41.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 3D | COCO test-dev | AP50 | 67.9 | DCNv2 (ResNet-101, multi-scale) |
| 3D | COCO test-dev | AP75 | 50.8 | DCNv2 (ResNet-101, multi-scale) |
| 3D | COCO test-dev | APL | 59.5 | DCNv2 (ResNet-101, multi-scale) |
| 3D | COCO test-dev | APM | 49.1 | DCNv2 (ResNet-101, multi-scale) |
| 3D | COCO test-dev | APS | 27.8 | DCNv2 (ResNet-101, multi-scale) |
| 3D | COCO test-dev | box mAP | 46 | DCNv2 (ResNet-101, multi-scale) |
| 3D | COCO minival | box AP | 43.1 | Mask R-CNN (ResNet-101, DCNv2) |
| 3D | COCO minival | APL | 58.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 3D | COCO minival | APM | 45.8 | Faster R-CNN (ResNet-101, DCNv2) |
| 3D | COCO minival | APS | 22.2 | Faster R-CNN (ResNet-101, DCNv2) |
| 3D | COCO minival | box AP | 41.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Classification | COCO test-dev | AP50 | 67.9 | DCNv2 (ResNet-101, multi-scale) |
| 2D Classification | COCO test-dev | AP75 | 50.8 | DCNv2 (ResNet-101, multi-scale) |
| 2D Classification | COCO test-dev | APL | 59.5 | DCNv2 (ResNet-101, multi-scale) |
| 2D Classification | COCO test-dev | APM | 49.1 | DCNv2 (ResNet-101, multi-scale) |
| 2D Classification | COCO test-dev | APS | 27.8 | DCNv2 (ResNet-101, multi-scale) |
| 2D Classification | COCO test-dev | box mAP | 46 | DCNv2 (ResNet-101, multi-scale) |
| 2D Classification | COCO minival | box AP | 43.1 | Mask R-CNN (ResNet-101, DCNv2) |
| 2D Classification | COCO minival | APL | 58.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Classification | COCO minival | APM | 45.8 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Classification | COCO minival | APS | 22.2 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Classification | COCO minival | box AP | 41.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Object Detection | COCO test-dev | AP50 | 67.9 | DCNv2 (ResNet-101, multi-scale) |
| 2D Object Detection | COCO test-dev | AP75 | 50.8 | DCNv2 (ResNet-101, multi-scale) |
| 2D Object Detection | COCO test-dev | APL | 59.5 | DCNv2 (ResNet-101, multi-scale) |
| 2D Object Detection | COCO test-dev | APM | 49.1 | DCNv2 (ResNet-101, multi-scale) |
| 2D Object Detection | COCO test-dev | APS | 27.8 | DCNv2 (ResNet-101, multi-scale) |
| 2D Object Detection | COCO test-dev | box mAP | 46 | DCNv2 (ResNet-101, multi-scale) |
| 2D Object Detection | COCO minival | box AP | 43.1 | Mask R-CNN (ResNet-101, DCNv2) |
| 2D Object Detection | COCO minival | APL | 58.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Object Detection | COCO minival | APM | 45.8 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Object Detection | COCO minival | APS | 22.2 | Faster R-CNN (ResNet-101, DCNv2) |
| 2D Object Detection | COCO minival | box AP | 41.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 16k | COCO test-dev | AP50 | 67.9 | DCNv2 (ResNet-101, multi-scale) |
| 16k | COCO test-dev | AP75 | 50.8 | DCNv2 (ResNet-101, multi-scale) |
| 16k | COCO test-dev | APL | 59.5 | DCNv2 (ResNet-101, multi-scale) |
| 16k | COCO test-dev | APM | 49.1 | DCNv2 (ResNet-101, multi-scale) |
| 16k | COCO test-dev | APS | 27.8 | DCNv2 (ResNet-101, multi-scale) |
| 16k | COCO test-dev | box mAP | 46 | DCNv2 (ResNet-101, multi-scale) |
| 16k | COCO minival | box AP | 43.1 | Mask R-CNN (ResNet-101, DCNv2) |
| 16k | COCO minival | APL | 58.7 | Faster R-CNN (ResNet-101, DCNv2) |
| 16k | COCO minival | APM | 45.8 | Faster R-CNN (ResNet-101, DCNv2) |
| 16k | COCO minival | APS | 22.2 | Faster R-CNN (ResNet-101, DCNv2) |
| 16k | COCO minival | box AP | 41.7 | Faster R-CNN (ResNet-101, DCNv2) |