Shengyu Huang, Zan Gojcic, Mikhail Usvyatsov, Andreas Wieser, Konrad Schindler
We introduce PREDATOR, a model for pairwise point-cloud registration with deep attention to the overlap region. Different from previous work, our model is specifically designed to handle (also) point-cloud pairs with low overlap. Its key novelty is an overlap-attention block for early information exchange between the latent encodings of the two point clouds. In this way the subsequent decoding of the latent representations into per-point features is conditioned on the respective other point cloud, and thus can predict which points are not only salient, but also lie in the overlap region between the two point clouds. The ability to focus on points that are relevant for matching greatly improves performance: PREDATOR raises the rate of successful registrations by more than 20% in the low-overlap scenario, and also sets a new state of the art for the 3DMatch benchmark with 89% registration recall.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Point Cloud Registration | 3DLoMatch (10-30% overlap) | Recall ( correspondence RMSE below 0.2) | 62.5 | Predator-1k |
| Point Cloud Registration | 3DLoMatch (10-30% overlap) | Recall ( correspondence RMSE below 0.2) | 59.8 | Predator-5k |
| Point Cloud Registration | 3DLoMatch (10-30% overlap) | Recall ( correspondence RMSE below 0.2) | 24 | Predator-NR |
| Point Cloud Registration | KITTI (trained on 3DMatch) | Success Rate | 41.2 | Predator |
| Point Cloud Registration | ScanNet++ (trained on 3DMatch) | Recall ( correspondence RMSE below 0.2) | 63.7 | Predator |
| Point Cloud Registration | 3DMatch (at least 30% overlapped - sample 5k interest points) | Recall ( correspondence RMSE below 0.2) | 90.5 | Predator-1k |
| Point Cloud Registration | 3DMatch (at least 30% overlapped - sample 5k interest points) | Recall ( correspondence RMSE below 0.2) | 89 | Predator-5k |
| Point Cloud Registration | 3DMatch (at least 30% overlapped - sample 5k interest points) | Recall ( correspondence RMSE below 0.2) | 62.7 | Predator-NR |
| Point Cloud Registration | RotKITTI Registration Benchmark | RR@(1,0.1) | 35 | PREDATOR |
| Point Cloud Registration | RotKITTI Registration Benchmark | RR@(1.5,0.3) | 41.6 | PREDATOR |
| 3D Point Cloud Interpolation | 3DLoMatch (10-30% overlap) | Recall ( correspondence RMSE below 0.2) | 62.5 | Predator-1k |
| 3D Point Cloud Interpolation | 3DLoMatch (10-30% overlap) | Recall ( correspondence RMSE below 0.2) | 59.8 | Predator-5k |
| 3D Point Cloud Interpolation | 3DLoMatch (10-30% overlap) | Recall ( correspondence RMSE below 0.2) | 24 | Predator-NR |
| 3D Point Cloud Interpolation | KITTI (trained on 3DMatch) | Success Rate | 41.2 | Predator |
| 3D Point Cloud Interpolation | ScanNet++ (trained on 3DMatch) | Recall ( correspondence RMSE below 0.2) | 63.7 | Predator |
| 3D Point Cloud Interpolation | 3DMatch (at least 30% overlapped - sample 5k interest points) | Recall ( correspondence RMSE below 0.2) | 90.5 | Predator-1k |
| 3D Point Cloud Interpolation | 3DMatch (at least 30% overlapped - sample 5k interest points) | Recall ( correspondence RMSE below 0.2) | 89 | Predator-5k |
| 3D Point Cloud Interpolation | 3DMatch (at least 30% overlapped - sample 5k interest points) | Recall ( correspondence RMSE below 0.2) | 62.7 | Predator-NR |
| 3D Point Cloud Interpolation | RotKITTI Registration Benchmark | RR@(1,0.1) | 35 | PREDATOR |
| 3D Point Cloud Interpolation | RotKITTI Registration Benchmark | RR@(1.5,0.3) | 41.6 | PREDATOR |