Denis Tome, Chris Russell, Lourdes Agapito
We propose a unified formulation for the problem of 3D human pose estimation from a single raw RGB image that reasons jointly about 2D joint estimation and 3D pose reconstruction to improve both tasks. We take an integrated approach that fuses probabilistic knowledge of 3D human pose with a multi-stage CNN architecture and uses the knowledge of plausible 3D landmark locations to refine the search for better 2D locations. The entire process is trained end-to-end, is extremely efficient and obtains state- of-the-art results on Human3.6M outperforming previous approaches both on 2D and 3D errors.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| 3D Human Pose Estimation | Human3.6M | Average MPJPE (mm) | 88.39 | Projected-pose belief maps + 2D fusion layers |
| 3D Human Pose Estimation | Human3.6M | Frames Needed | 1 | Projected-pose belief maps + 2D fusion layers |
| 3D Human Pose Estimation | Human3.6M | Average MPJPE (mm) | 88.4 | Tome et al. |
| 3D Human Pose Estimation | Human3.6M | Number of Frames Per View | 1 | Tome et al. |
| 3D Human Pose Estimation | Human3.6M | Number of Views | 1 | Tome et al. |
| Pose Estimation | Human3.6M | Average MPJPE (mm) | 88.39 | Projected-pose belief maps + 2D fusion layers |
| Pose Estimation | Human3.6M | Frames Needed | 1 | Projected-pose belief maps + 2D fusion layers |
| Pose Estimation | Human3.6M | Average MPJPE (mm) | 88.4 | Tome et al. |
| Pose Estimation | Human3.6M | Number of Frames Per View | 1 | Tome et al. |
| Pose Estimation | Human3.6M | Number of Views | 1 | Tome et al. |
| 3D | Human3.6M | Average MPJPE (mm) | 88.39 | Projected-pose belief maps + 2D fusion layers |
| 3D | Human3.6M | Frames Needed | 1 | Projected-pose belief maps + 2D fusion layers |
| 3D | Human3.6M | Average MPJPE (mm) | 88.4 | Tome et al. |
| 3D | Human3.6M | Number of Frames Per View | 1 | Tome et al. |
| 3D | Human3.6M | Number of Views | 1 | Tome et al. |
| 1 Image, 2*2 Stitchi | Human3.6M | Average MPJPE (mm) | 88.39 | Projected-pose belief maps + 2D fusion layers |
| 1 Image, 2*2 Stitchi | Human3.6M | Frames Needed | 1 | Projected-pose belief maps + 2D fusion layers |
| 1 Image, 2*2 Stitchi | Human3.6M | Average MPJPE (mm) | 88.4 | Tome et al. |
| 1 Image, 2*2 Stitchi | Human3.6M | Number of Frames Per View | 1 | Tome et al. |
| 1 Image, 2*2 Stitchi | Human3.6M | Number of Views | 1 | Tome et al. |