Benjamin Kiefer, David Ott, Andreas Zell
Acquiring data to train deep learning-based object detectors on Unmanned Aerial Vehicles (UAVs) is expensive, time-consuming and may even be prohibited by law in specific environments. On the other hand, synthetic data is fast and cheap to access. In this work, we explore the potential use of synthetic data in object detection from UAVs across various application environments. For that, we extend the open-source framework DeepGTAV to work for UAV scenarios. We capture various large-scale high-resolution synthetic data sets in several domains to demonstrate their use in real-world object detection from UAVs by analyzing multiple training strategies across several models. Furthermore, we analyze several different data generation and sampling parameters to provide actionable engineering advice for further scientific research. The DeepGTAV framework is available at https://git.io/Jyf5j.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Object Detection | SeaDronesSee | mAP@0.5 | 59.2 | Synth Pretrained Faster R-CNN ResNeXt-101-FPN |
| Object Detection | SeaDronesSee | mAP@0.5 | 59.08 | Synth Pretrained Yolo5 |
| Object Detection | SeaDronesSee | mAP@0.5 | 38.74 | Synth Pretrained EffDetD0 |
| Object Detection | SeaDronesSee | mAP@0.50 | 54.74 | Yolo 5 |
| 3D | SeaDronesSee | mAP@0.5 | 59.2 | Synth Pretrained Faster R-CNN ResNeXt-101-FPN |
| 3D | SeaDronesSee | mAP@0.5 | 59.08 | Synth Pretrained Yolo5 |
| 3D | SeaDronesSee | mAP@0.5 | 38.74 | Synth Pretrained EffDetD0 |
| 3D | SeaDronesSee | mAP@0.50 | 54.74 | Yolo 5 |
| 2D Classification | SeaDronesSee | mAP@0.5 | 59.2 | Synth Pretrained Faster R-CNN ResNeXt-101-FPN |
| 2D Classification | SeaDronesSee | mAP@0.5 | 59.08 | Synth Pretrained Yolo5 |
| 2D Classification | SeaDronesSee | mAP@0.5 | 38.74 | Synth Pretrained EffDetD0 |
| 2D Classification | SeaDronesSee | mAP@0.50 | 54.74 | Yolo 5 |
| 2D Object Detection | SeaDronesSee | mAP@0.5 | 59.2 | Synth Pretrained Faster R-CNN ResNeXt-101-FPN |
| 2D Object Detection | SeaDronesSee | mAP@0.5 | 59.08 | Synth Pretrained Yolo5 |
| 2D Object Detection | SeaDronesSee | mAP@0.5 | 38.74 | Synth Pretrained EffDetD0 |
| 2D Object Detection | SeaDronesSee | mAP@0.50 | 54.74 | Yolo 5 |
| 16k | SeaDronesSee | mAP@0.5 | 59.2 | Synth Pretrained Faster R-CNN ResNeXt-101-FPN |
| 16k | SeaDronesSee | mAP@0.5 | 59.08 | Synth Pretrained Yolo5 |
| 16k | SeaDronesSee | mAP@0.5 | 38.74 | Synth Pretrained EffDetD0 |
| 16k | SeaDronesSee | mAP@0.50 | 54.74 | Yolo 5 |