Roman Klokov, Victor Lempitsky
We present a new deep learning architecture (called Kd-network) that is designed for 3D model recognition tasks and works with unstructured point clouds. The new architecture performs multiplicative transformations and share parameters of these transformations according to the subdivisions of the point clouds imposed onto them by Kd-trees. Unlike the currently dominant convolutional architectures that usually require rasterization on uniform two-dimensional or three-dimensional grids, Kd-networks do not rely on such grids in any way and therefore avoid poor scaling behaviour. In a series of experiments with popular shape recognition benchmarks, Kd-networks demonstrate competitive performance in a number of shape recognition tasks such as shape classification, shape retrieval and shape part segmentation.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Semantic Segmentation | ShapeNet-Part | Class Average IoU | 77.4 | Kd-net |
| Semantic Segmentation | ShapeNet-Part | Instance Average IoU | 82.3 | Kd-net |
| Shape Representation Of 3D Point Clouds | ModelNet40 | Overall Accuracy | 91.8 | Kd-Net |
| Shape Representation Of 3D Point Clouds | ModelNet40 | Overall Accuracy | 90.6 | Kd-net |
| 3D Point Cloud Classification | ModelNet40 | Overall Accuracy | 91.8 | Kd-Net |
| 3D Point Cloud Classification | ModelNet40 | Overall Accuracy | 90.6 | Kd-net |
| 10-shot image generation | ShapeNet-Part | Class Average IoU | 77.4 | Kd-net |
| 10-shot image generation | ShapeNet-Part | Instance Average IoU | 82.3 | Kd-net |
| 3D Point Cloud Reconstruction | ModelNet40 | Overall Accuracy | 91.8 | Kd-Net |
| 3D Point Cloud Reconstruction | ModelNet40 | Overall Accuracy | 90.6 | Kd-net |