Bum Jun Kim, Hyeonah Jang, Sang Woo Kim
The latest advances in deep learning have facilitated the development of highly accurate monocular depth estimation models. However, when training a monocular depth estimation network, practitioners and researchers have observed not a number (NaN) loss, which disrupts gradient descent optimization. Although several practitioners have reported the stochastic and mysterious occurrence of NaN loss that bothers training, its root cause is not discussed in the literature. This study conducted an in-depth analysis of NaN loss during training a monocular depth estimation network and identified three types of vulnerabilities that cause NaN loss: 1) the use of square root loss, which leads to an unstable gradient; 2) the log-sigmoid function, which exhibits numerical stability issues; and 3) certain variance implementations, which yield incorrect computations. Furthermore, for each vulnerability, the occurrence of NaN loss was demonstrated and practical guidelines to prevent NaN loss were presented. Experiments showed that both optimization stability and performance on monocular depth estimation could be improved by following our guidelines.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Depth Estimation | NYU-Depth V2 | Delta < 1.25 | 0.9361 | MIM-Swin-V2 |
| Depth Estimation | NYU-Depth V2 | Delta < 1.25^2 | 0.9916 | MIM-Swin-V2 |
| Depth Estimation | NYU-Depth V2 | Delta < 1.25^3 | 0.9981 | MIM-Swin-V2 |
| Depth Estimation | NYU-Depth V2 | RMSE | 0.3046 | MIM-Swin-V2 |
| Depth Estimation | NYU-Depth V2 | absolute relative error | 0.0864 | MIM-Swin-V2 |
| Depth Estimation | NYU-Depth V2 | log 10 | 0.0365 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | Delta < 1.25 | 0.9757 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | Delta < 1.25^2 | 0.9974 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | Delta < 1.25^3 | 0.9994 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | RMSE | 2.0373 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | RMSE log | 0.077 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | Sq Rel | 0.1458 | MIM-Swin-V2 |
| Depth Estimation | KITTI Eigen split | absolute relative error | 0.0508 | MIM-Swin-V2 |
| 3D | NYU-Depth V2 | Delta < 1.25 | 0.9361 | MIM-Swin-V2 |
| 3D | NYU-Depth V2 | Delta < 1.25^2 | 0.9916 | MIM-Swin-V2 |
| 3D | NYU-Depth V2 | Delta < 1.25^3 | 0.9981 | MIM-Swin-V2 |
| 3D | NYU-Depth V2 | RMSE | 0.3046 | MIM-Swin-V2 |
| 3D | NYU-Depth V2 | absolute relative error | 0.0864 | MIM-Swin-V2 |
| 3D | NYU-Depth V2 | log 10 | 0.0365 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | Delta < 1.25 | 0.9757 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | Delta < 1.25^2 | 0.9974 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | Delta < 1.25^3 | 0.9994 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | RMSE | 2.0373 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | RMSE log | 0.077 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | Sq Rel | 0.1458 | MIM-Swin-V2 |
| 3D | KITTI Eigen split | absolute relative error | 0.0508 | MIM-Swin-V2 |