Shai Aharon, Gil Ben-Artzi
Adaptive image restoration models can restore images with different degradation levels at inference time without the need to retrain the model. We present an approach that is highly accurate and allows a significant reduction in the number of parameters. In contrast to existing methods, our approach can restore images using a single fixed-size model, regardless of the number of degradation levels. On popular datasets, our approach yields state-of-the-art results in terms of size and accuracy for a variety of image restoration tasks, including denoising, deJPEG, and super-resolution.
| Task | Dataset | Metric | Value | Model |
|---|---|---|---|---|
| Super-Resolution | Set5 - 6x upscaling | PSNR | 24.92 | HyperRes |
| Super-Resolution | Set5 - 6x upscaling | SSIM | 0.71 | HyperRes |
| Super-Resolution | Set5 - 5x upscaling | PSNR | 25.63 | HyperRes |
| Super-Resolution | Set5 - 5x upscaling | SSIM | 0.74 | HyperRes |
| Super-Resolution | Set5 - 3x upscaling | PSNR | 29.77 | HyperRes |
| Super-Resolution | Set5 - 3x upscaling | SSIM | 0.84 | HyperRes |
| Super-Resolution | Set5 - 2x upscaling | PSNR | 36.69 | HyperRes |
| Super-Resolution | Set5 - 2x upscaling | SSIM | 0.94 | HyperRes |
| Image Restoration | Live1 (Quality 10 Grayscale) | PSNR | 28.81 | HyperRes |
| Image Restoration | Live1 (Quality 10 Grayscale) | SSIM | 0.82 | HyperRes |
| Image Restoration | Live1 (Quality 50 Grayscale) | PSNR | 34.39 | HyperRes |
| Image Restoration | Live1 (Quality 50 Grayscale) | SSIM | 0.94 | HyperRes |
| Image Restoration | Live1 (Quality 70 Grayscale) | PSNR | 36.38 | HyperRes |
| Image Restoration | Live1 (Quality 70 Grayscale) | SSIM | 0.96 | HyperRes |
| Image Restoration | LIVE1 (Quality 30 Grayscale) | PSNR | 32.56 | HyperRes |
| Image Restoration | LIVE1 (Quality 30 Grayscale) | SSIM | 0.91 | HyperRes |
| Image Restoration | Live1 (Quality 80 Grayscale) | PSNR | 38.09 | HyperRes |
| Image Restoration | Live1 (Quality 80 Grayscale) | SSIM | 0.97 | HyperRes |
| Denoising | CBSD68 sigma5 | PSNR | 39.96 | HyperRes |
| Denoising | CBSD68 sigma5 | SSIM | 0.98 | HyperRes |
| Denoising | CBSD68 sigma15 | PSNR | 34.02 | HyperRes |
| Denoising | CBSD68 sigma15 | SSIM | 0.95 | HyperRes |
| Denoising | CBSD68 sigma45 | PSNR | 28.64 | HyperRes |
| Denoising | CBSD68 sigma45 | SSIM | 0.81 | HyperRes |
| Denoising | CBSD68 sigma65 | PSNR | 27.06 | HyperRes |
| Denoising | CBSD68 sigma65 | SSIM | 0.76 | HyperRes |
| Denoising | CBSD68 sigma25 | PSNR | 31.41 | HyperRes |
| Denoising | CBSD68 sigma25 | SSIM | 0.89 | HyperRes |
| Denoising | CBSD68 sigma35 | PSNR | 29.8 | HyperRes |
| Denoising | CBSD68 sigma35 | SSIM | 0.85 | HyperRes |
| Denoising | CBSD68 sigma55 | PSNR | 27.78 | HyperRes |
| Denoising | CBSD68 sigma55 | SSIM | 0.78 | HyperRes |
| Denoising | CBSD68 sigma75 | PSNR | 26.46 | HyperRes |
| Denoising | CBSD68 sigma75 | SSIM | 0.73 | HyperRes |
| Denoising | CBSD68 sigma85 | PSNR | 25.95 | HyperRes |
| Denoising | CBSD68 sigma85 | SSIM | 0.71 | HyperRes |
| Image Super-Resolution | Set5 - 6x upscaling | PSNR | 24.92 | HyperRes |
| Image Super-Resolution | Set5 - 6x upscaling | SSIM | 0.71 | HyperRes |
| Image Super-Resolution | Set5 - 5x upscaling | PSNR | 25.63 | HyperRes |
| Image Super-Resolution | Set5 - 5x upscaling | SSIM | 0.74 | HyperRes |
| Image Super-Resolution | Set5 - 3x upscaling | PSNR | 29.77 | HyperRes |
| Image Super-Resolution | Set5 - 3x upscaling | SSIM | 0.84 | HyperRes |
| Image Super-Resolution | Set5 - 2x upscaling | PSNR | 36.69 | HyperRes |
| Image Super-Resolution | Set5 - 2x upscaling | SSIM | 0.94 | HyperRes |
| 3D Architecture | CBSD68 sigma5 | PSNR | 39.96 | HyperRes |
| 3D Architecture | CBSD68 sigma5 | SSIM | 0.98 | HyperRes |
| 3D Architecture | CBSD68 sigma15 | PSNR | 34.02 | HyperRes |
| 3D Architecture | CBSD68 sigma15 | SSIM | 0.95 | HyperRes |
| 3D Architecture | CBSD68 sigma45 | PSNR | 28.64 | HyperRes |
| 3D Architecture | CBSD68 sigma45 | SSIM | 0.81 | HyperRes |
| 3D Architecture | CBSD68 sigma65 | PSNR | 27.06 | HyperRes |
| 3D Architecture | CBSD68 sigma65 | SSIM | 0.76 | HyperRes |
| 3D Architecture | CBSD68 sigma25 | PSNR | 31.41 | HyperRes |
| 3D Architecture | CBSD68 sigma25 | SSIM | 0.89 | HyperRes |
| 3D Architecture | CBSD68 sigma35 | PSNR | 29.8 | HyperRes |
| 3D Architecture | CBSD68 sigma35 | SSIM | 0.85 | HyperRes |
| 3D Architecture | CBSD68 sigma55 | PSNR | 27.78 | HyperRes |
| 3D Architecture | CBSD68 sigma55 | SSIM | 0.78 | HyperRes |
| 3D Architecture | CBSD68 sigma75 | PSNR | 26.46 | HyperRes |
| 3D Architecture | CBSD68 sigma75 | SSIM | 0.73 | HyperRes |
| 3D Architecture | CBSD68 sigma85 | PSNR | 25.95 | HyperRes |
| 3D Architecture | CBSD68 sigma85 | SSIM | 0.71 | HyperRes |
| 10-shot image generation | Live1 (Quality 10 Grayscale) | PSNR | 28.81 | HyperRes |
| 10-shot image generation | Live1 (Quality 10 Grayscale) | SSIM | 0.82 | HyperRes |
| 10-shot image generation | Live1 (Quality 50 Grayscale) | PSNR | 34.39 | HyperRes |
| 10-shot image generation | Live1 (Quality 50 Grayscale) | SSIM | 0.94 | HyperRes |
| 10-shot image generation | Live1 (Quality 70 Grayscale) | PSNR | 36.38 | HyperRes |
| 10-shot image generation | Live1 (Quality 70 Grayscale) | SSIM | 0.96 | HyperRes |
| 10-shot image generation | LIVE1 (Quality 30 Grayscale) | PSNR | 32.56 | HyperRes |
| 10-shot image generation | LIVE1 (Quality 30 Grayscale) | SSIM | 0.91 | HyperRes |
| 10-shot image generation | Live1 (Quality 80 Grayscale) | PSNR | 38.09 | HyperRes |
| 10-shot image generation | Live1 (Quality 80 Grayscale) | SSIM | 0.97 | HyperRes |
| 3D Object Super-Resolution | Set5 - 6x upscaling | PSNR | 24.92 | HyperRes |
| 3D Object Super-Resolution | Set5 - 6x upscaling | SSIM | 0.71 | HyperRes |
| 3D Object Super-Resolution | Set5 - 5x upscaling | PSNR | 25.63 | HyperRes |
| 3D Object Super-Resolution | Set5 - 5x upscaling | SSIM | 0.74 | HyperRes |
| 3D Object Super-Resolution | Set5 - 3x upscaling | PSNR | 29.77 | HyperRes |
| 3D Object Super-Resolution | Set5 - 3x upscaling | SSIM | 0.84 | HyperRes |
| 3D Object Super-Resolution | Set5 - 2x upscaling | PSNR | 36.69 | HyperRes |
| 3D Object Super-Resolution | Set5 - 2x upscaling | SSIM | 0.94 | HyperRes |
| 16k | Set5 - 6x upscaling | PSNR | 24.92 | HyperRes |
| 16k | Set5 - 6x upscaling | SSIM | 0.71 | HyperRes |
| 16k | Set5 - 5x upscaling | PSNR | 25.63 | HyperRes |
| 16k | Set5 - 5x upscaling | SSIM | 0.74 | HyperRes |
| 16k | Set5 - 3x upscaling | PSNR | 29.77 | HyperRes |
| 16k | Set5 - 3x upscaling | SSIM | 0.84 | HyperRes |
| 16k | Set5 - 2x upscaling | PSNR | 36.69 | HyperRes |
| 16k | Set5 - 2x upscaling | SSIM | 0.94 | HyperRes |