Haoxi Ran, Jun Liu, Chengjie Wang
Most prior work represents the shapes of point clouds by coordinates. However, it is insufficient to describe the local geometry directly. In this paper, we present \textbf{RepSurf} (representative surfaces), a novel representation of point clouds to \textbf{explicitly} depict the very local structure. We explore two variants of RepSurf, Triangular RepSurf and Umbrella RepSurf inspired by triangle meshes and umbrella curvature in computer graphics. We compute the representations of RepSurf by predefined geometric priors after surface reconstruction. RepSurf can be a plug-and-play module for most point cloud models thanks to its free collaboration with irregular points. Based on a simple baseline of PointNet++ (SSG version), Umbrella RepSurf surpasses the previous state-of-the-art by a large margin for classification, segmentation and detection on various benchmarks in terms of performance and efficiency. With an increase of around \textbf{0.008M} number of parameters, \textbf{0.04G} FLOPs, and \textbf{1.12ms} inference time, our method achieves \textbf{94.7\%} (+0.5\%) on ModelNet40, and \textbf{84.6\%} (+1.8\%) on ScanObjectNN for classification, while \textbf{74.3\%} (+0.8\%) mIoU on S3DIS 6-fold, and \textbf{70.0\%} (+1.6\%) mIoU on ScanNet for segmentation. For detection, previous state-of-the-art detector with our RepSurf obtains \textbf{71.2\%} (+2.1\%) mAP$\mathit{_{25}}$, \textbf{54.8\%} (+2.0\%) mAP$\mathit{_{50}}$ on ScanNetV2, and \textbf{64.9\%} (+1.9\%) mAP$\mathit{_{25}}$, \textbf{47.7\%} (+2.5\%) mAP$\mathit{_{50}}$ on SUN RGB-D. Our lightweight Triangular RepSurf performs its excellence on these benchmarks as well. The code is publicly available at \url{https://github.com/hancyran/RepSurf}.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Semantic Segmentation | S3DIS Area5 | mAcc | 76 | RepSurf-U |
| Semantic Segmentation | S3DIS Area5 | mIoU | 68.9 | RepSurf-U |
| Semantic Segmentation | S3DIS Area5 | oAcc | 90.2 | RepSurf-U |
| Semantic Segmentation | S3DIS | Mean IoU | 74.3 | RepSurf-U |
| Semantic Segmentation | S3DIS | Params (M) | 0.97 | RepSurf-U |
| Semantic Segmentation | S3DIS | mAcc | 82.6 | RepSurf-U |
| Semantic Segmentation | S3DIS | oAcc | 90.8 | RepSurf-U |
| Object Detection | SUN-RGBD val | mAP@0.25 | 64.9 | RepSurf-U |
| Object Detection | SUN-RGBD val | mAP@0.5 | 47.7 | RepSurf-U |
| Object Detection | ScanNetV2 | mAP@0.25 | 71.2 | RepSurf-U |
| Object Detection | ScanNetV2 | mAP@0.5 | 54.8 | RepSurf-U |
| 3D | SUN-RGBD val | mAP@0.25 | 64.9 | RepSurf-U |
| 3D | SUN-RGBD val | mAP@0.5 | 47.7 | RepSurf-U |
| 3D | ScanNetV2 | mAP@0.25 | 71.2 | RepSurf-U |
| 3D | ScanNetV2 | mAP@0.5 | 54.8 | RepSurf-U |
| Shape Representation Of 3D Point Clouds | ScanObjectNN | Overall Accuracy | 86 | RepSurf-U (2x) |
| Shape Representation Of 3D Point Clouds | ScanObjectNN | Overall Accuracy | 84.6 | RepSurf-U |
| Shape Representation Of 3D Point Clouds | ModelNet40 | Overall Accuracy | 94.7 | RepSurf-U |
| 3D Object Detection | SUN-RGBD val | mAP@0.25 | 64.9 | RepSurf-U |
| 3D Object Detection | SUN-RGBD val | mAP@0.5 | 47.7 | RepSurf-U |
| 3D Object Detection | ScanNetV2 | mAP@0.25 | 71.2 | RepSurf-U |
| 3D Object Detection | ScanNetV2 | mAP@0.5 | 54.8 | RepSurf-U |
| 3D Point Cloud Classification | ScanObjectNN | Overall Accuracy | 86 | RepSurf-U (2x) |
| 3D Point Cloud Classification | ScanObjectNN | Overall Accuracy | 84.6 | RepSurf-U |
| 3D Point Cloud Classification | ModelNet40 | Overall Accuracy | 94.7 | RepSurf-U |
| 2D Classification | SUN-RGBD val | mAP@0.25 | 64.9 | RepSurf-U |
| 2D Classification | SUN-RGBD val | mAP@0.5 | 47.7 | RepSurf-U |
| 2D Classification | ScanNetV2 | mAP@0.25 | 71.2 | RepSurf-U |
| 2D Classification | ScanNetV2 | mAP@0.5 | 54.8 | RepSurf-U |
| 2D Object Detection | SUN-RGBD val | mAP@0.25 | 64.9 | RepSurf-U |
| 2D Object Detection | SUN-RGBD val | mAP@0.5 | 47.7 | RepSurf-U |
| 2D Object Detection | ScanNetV2 | mAP@0.25 | 71.2 | RepSurf-U |
| 2D Object Detection | ScanNetV2 | mAP@0.5 | 54.8 | RepSurf-U |
| 10-shot image generation | S3DIS Area5 | mAcc | 76 | RepSurf-U |
| 10-shot image generation | S3DIS Area5 | mIoU | 68.9 | RepSurf-U |
| 10-shot image generation | S3DIS Area5 | oAcc | 90.2 | RepSurf-U |
| 10-shot image generation | S3DIS | Mean IoU | 74.3 | RepSurf-U |
| 10-shot image generation | S3DIS | Params (M) | 0.97 | RepSurf-U |
| 10-shot image generation | S3DIS | mAcc | 82.6 | RepSurf-U |
| 10-shot image generation | S3DIS | oAcc | 90.8 | RepSurf-U |
| 3D Point Cloud Reconstruction | ScanObjectNN | Overall Accuracy | 86 | RepSurf-U (2x) |
| 3D Point Cloud Reconstruction | ScanObjectNN | Overall Accuracy | 84.6 | RepSurf-U |
| 3D Point Cloud Reconstruction | ModelNet40 | Overall Accuracy | 94.7 | RepSurf-U |
| 16k | SUN-RGBD val | mAP@0.25 | 64.9 | RepSurf-U |
| 16k | SUN-RGBD val | mAP@0.5 | 47.7 | RepSurf-U |
| 16k | ScanNetV2 | mAP@0.25 | 71.2 | RepSurf-U |
| 16k | ScanNetV2 | mAP@0.5 | 54.8 | RepSurf-U |