Tao Jiang, Peng Lu, Li Zhang, Ningsheng Ma, Rui Han, Chengqi Lyu, Yining Li, Kai Chen
Recent studies on 2D pose estimation have achieved excellent performance on public benchmarks, yet its application in the industrial community still suffers from heavy model parameters and high latency. In order to bridge this gap, we empirically explore key factors in pose estimation including paradigm, model architecture, training strategy, and deployment, and present a high-performance real-time multi-person pose estimation framework, RTMPose, based on MMPose. Our RTMPose-m achieves 75.8% AP on COCO with 90+ FPS on an Intel i7-11700 CPU and 430+ FPS on an NVIDIA GTX 1660 Ti GPU, and RTMPose-l achieves 67.0% AP on COCO-WholeBody with 130+ FPS. To further evaluate RTMPose's capability in critical real-time applications, we also report the performance after deploying on the mobile device. Our RTMPose-s achieves 72.2% AP on COCO with 70+ FPS on a Snapdragon 865 chip, outperforming existing open-source libraries. Code and models are released at https://github.com/open-mmlab/mmpose/tree/1.x/projects/rtmpose.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Pose Estimation | OCHuman | Test AP | 80.3 | RTMPose(RTMPose-l, GT bounding boxes) |
| Pose Estimation | OCHuman | Validation AP | 80.5 | RTMPose(RTMPose-l, GT bounding boxes) |
| 3D | OCHuman | Test AP | 80.3 | RTMPose(RTMPose-l, GT bounding boxes) |
| 3D | OCHuman | Validation AP | 80.5 | RTMPose(RTMPose-l, GT bounding boxes) |
| 2D Human Pose Estimation | Human-Art | AP | 0.311 | RTMPose-s |
| 2D Human Pose Estimation | Human-Art | AP (gt bbox) | 0.753 | RTMPose-l |
| 2D Human Pose Estimation | Human-Art | Validation AP | 83.5 | RTMPose-l |
| 2D Human Pose Estimation | COCO-WholeBody | WB | 65.3 | RTMPose |
| 2D Human Pose Estimation | COCO-WholeBody | body | 71.4 | RTMPose |
| 2D Human Pose Estimation | COCO-WholeBody | face | 88.9 | RTMPose |
| 2D Human Pose Estimation | COCO-WholeBody | foot | 69.2 | RTMPose |
| 2D Human Pose Estimation | COCO-WholeBody | hand | 59 | RTMPose |
| 1 Image, 2*2 Stitchi | OCHuman | Test AP | 80.3 | RTMPose(RTMPose-l, GT bounding boxes) |
| 1 Image, 2*2 Stitchi | OCHuman | Validation AP | 80.5 | RTMPose(RTMPose-l, GT bounding boxes) |