Adam Van Etten, Dave Lindenbaum, Todd M. Bacastow
Foundational mapping remains a challenge in many parts of the world, particularly in dynamic scenarios such as natural disasters when timely updates are critical. Updating maps is currently a highly manual process requiring a large number of human labelers to either create features or rigorously validate automated outputs. We propose that the frequent revisits of earth imaging satellite constellations may accelerate existing efforts to quickly update foundational maps when combined with advanced machine learning techniques. Accordingly, the SpaceNet partners (CosmiQ Works, Radiant Solutions, and NVIDIA), released a large corpus of labeled satellite imagery on Amazon Web Services (AWS) called SpaceNet. The SpaceNet partners also launched a series of public prize competitions to encourage improvement of remote sensing machine learning algorithms. The first two of these competitions focused on automated building footprint extraction, and the most recent challenge focused on road network extraction. In this paper we discuss the SpaceNet imagery, labels, evaluation metrics, prize challenge results to date, and future plans for the SpaceNet challenge series.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Object Detection | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.6 | YOLT |
| Object Detection | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.57 | Multi-Task Network Cascades |
| Object Detection | SpaceNet 1 | F1 Score | 0.21 | YOLT |
| 3D | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.6 | YOLT |
| 3D | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.57 | Multi-Task Network Cascades |
| 3D | SpaceNet 1 | F1 Score | 0.21 | YOLT |
| 2D Classification | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.6 | YOLT |
| 2D Classification | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.57 | Multi-Task Network Cascades |
| 2D Classification | SpaceNet 1 | F1 Score | 0.21 | YOLT |
| 2D Object Detection | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.6 | YOLT |
| 2D Object Detection | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.57 | Multi-Task Network Cascades |
| 2D Object Detection | SpaceNet 1 | F1 Score | 0.21 | YOLT |
| 16k | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.6 | YOLT |
| 16k | SpaceNet 2 | F1 Score (Avg. over Cities) | 0.57 | Multi-Task Network Cascades |
| 16k | SpaceNet 1 | F1 Score | 0.21 | YOLT |