Jiaxin Li, Ben M. Chen, Gim Hee Lee
This paper presents SO-Net, a permutation invariant architecture for deep learning with orderless point clouds. The SO-Net models the spatial distribution of point cloud by building a Self-Organizing Map (SOM). Based on the SOM, SO-Net performs hierarchical feature extraction on individual points and SOM nodes, and ultimately represents the input point cloud by a single feature vector. The receptive field of the network can be systematically adjusted by conducting point-to-node k nearest neighbor search. In recognition tasks such as point cloud reconstruction, classification, object part segmentation and shape retrieval, our proposed network demonstrates performance that is similar with or better than state-of-the-art approaches. In addition, the training speed is significantly faster than existing point cloud recognition networks because of the parallelizability and simplicity of the proposed architecture. Our code is available at the project website. https://github.com/lijx10/SO-Net
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Semantic Segmentation | IntrA | DSC (A) | 88.76 | SO-Net |
| Semantic Segmentation | IntrA | DSC (V) | 97.09 | SO-Net |
| Semantic Segmentation | IntrA | IoU (A) | 81.4 | SO-Net |
| Semantic Segmentation | IntrA | IoU (V) | 94.46 | SO-Net |
| Semantic Segmentation | ShapeNet-Part | Instance Average IoU | 84.9 | SO-Net |
| Shape Representation Of 3D Point Clouds | IntrA | F1 score (5-fold) | 0.868 | SO-Net |
| Shape Representation Of 3D Point Clouds | ModelNet40 | Overall Accuracy | 90.9 | SO-Net |
| 3D Point Cloud Classification | IntrA | F1 score (5-fold) | 0.868 | SO-Net |
| 3D Point Cloud Classification | ModelNet40 | Overall Accuracy | 90.9 | SO-Net |
| 3D Point Cloud Linear Classification | ModelNet40 | Overall Accuracy | 87.5 | SO-Net |
| 10-shot image generation | IntrA | DSC (A) | 88.76 | SO-Net |
| 10-shot image generation | IntrA | DSC (V) | 97.09 | SO-Net |
| 10-shot image generation | IntrA | IoU (A) | 81.4 | SO-Net |
| 10-shot image generation | IntrA | IoU (V) | 94.46 | SO-Net |
| 10-shot image generation | ShapeNet-Part | Instance Average IoU | 84.9 | SO-Net |
| 3D Point Cloud Reconstruction | IntrA | F1 score (5-fold) | 0.868 | SO-Net |
| 3D Point Cloud Reconstruction | ModelNet40 | Overall Accuracy | 90.9 | SO-Net |