ILIAS

ILIAS is a large-scale test dataset for evaluation on Instance-Level Image retrieval At Scale. It is designed to support future research in image-to-image and text-to-image retrieval for particular objects and serves as a benchmark for evaluating representations of foundation or customized vision and vision-language models, as well as specialized retrieval techniques.

website | dataset | arxiv | huggingface

Composition

The dataset includes 1,000 object instances across diverse domains, with:

• 5,947 images in total:
  • 1,232 image queries, depicting query objects on clean or uniform background
  • 4,715 positive images, featuring the query objects in real-world conditions with clutter, occlusions, scale variations, and partial views
• 1,000 text queries, providing fine-grained textual descriptions of the query objects
• 100M distractors from YFCC100M to evaluate retrieval performance under large-scale settings, while asserting noise-free ground truth