Weiting Huang, Pengfei Ren, Jingyu Wang, Qi Qi, Haifeng Sun
In this paper, we propose an adaptive weighting regression (AWR) method to leverage the advantages of both detection-based and regression-based methods. Hand joint coordinates are estimated as discrete integration of all pixels in dense representation, guided by adaptive weight maps. This learnable aggregation process introduces both dense and joint supervision that allows end-to-end training and brings adaptability to weight maps, making the network more accurate and robust. Comprehensive exploration experiments are conducted to validate the effectiveness and generality of AWR under various experimental settings, especially its usefulness for different types of dense representation and input modality. Our method outperforms other state-of-the-art methods on four publicly available datasets, including NYU, ICVL, MSRA and HANDS 2017 dataset.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Hand | MSRA Hands | Average 3D Error | 7.15 | AWR |
| Hand | ICVL Hands | Average 3D Error | 5.98 | AWR |
| Hand | HANDS 2019 | Average 3D Error | 13.76 | AWR |
| Hand | NYU Hands | Average 3D Error | 7.48 | AWR |
| Hand | HANDS 2017 | Average 3D Error | 7.48 | AWR |
| Pose Estimation | MSRA Hands | Average 3D Error | 7.15 | AWR |
| Pose Estimation | ICVL Hands | Average 3D Error | 5.98 | AWR |
| Pose Estimation | HANDS 2019 | Average 3D Error | 13.76 | AWR |
| Pose Estimation | NYU Hands | Average 3D Error | 7.48 | AWR |
| Pose Estimation | HANDS 2017 | Average 3D Error | 7.48 | AWR |
| Hand Pose Estimation | MSRA Hands | Average 3D Error | 7.15 | AWR |
| Hand Pose Estimation | ICVL Hands | Average 3D Error | 5.98 | AWR |
| Hand Pose Estimation | HANDS 2019 | Average 3D Error | 13.76 | AWR |
| Hand Pose Estimation | NYU Hands | Average 3D Error | 7.48 | AWR |
| Hand Pose Estimation | HANDS 2017 | Average 3D Error | 7.48 | AWR |
| 3D | MSRA Hands | Average 3D Error | 7.15 | AWR |
| 3D | ICVL Hands | Average 3D Error | 5.98 | AWR |
| 3D | HANDS 2019 | Average 3D Error | 13.76 | AWR |
| 3D | NYU Hands | Average 3D Error | 7.48 | AWR |
| 3D | HANDS 2017 | Average 3D Error | 7.48 | AWR |
| 1 Image, 2*2 Stitchi | MSRA Hands | Average 3D Error | 7.15 | AWR |
| 1 Image, 2*2 Stitchi | ICVL Hands | Average 3D Error | 5.98 | AWR |
| 1 Image, 2*2 Stitchi | HANDS 2019 | Average 3D Error | 13.76 | AWR |
| 1 Image, 2*2 Stitchi | NYU Hands | Average 3D Error | 7.48 | AWR |
| 1 Image, 2*2 Stitchi | HANDS 2017 | Average 3D Error | 7.48 | AWR |