Visual Reasoning on WinoGAViL

Metric: Jaccard Index (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Sort:

#	Model↕	Jaccard Index▼	Extra Data	Paper	Date↕	Code
1	Humans	90	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
2	ViLT (Zero-Shot)	52	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
3	X-VLM (Zero-Shot)	46	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
4	CLIP-ViT-B/32 (Zero-Shot)	41	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
5	CLIP-ViT-L/14 (Zero-Shot)	40	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
6	CLIP-RN50x64/14 (Zero-Shot)	38	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
7	CLIP-RN50 (Zero-Shot)	35	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code
8	CLIP-ViL (Zero-Shot)	15	No	WinoGAViL: Gamified Association Benchmark to Cha...	2022-07-25	Code

#1HumansSOTA
90
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#2ViLT (Zero-Shot)
52
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#3X-VLM (Zero-Shot)
46
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#4CLIP-ViT-B/32 (Zero-Shot)
41
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#5CLIP-ViT-L/14 (Zero-Shot)
40
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#6CLIP-RN50x64/14 (Zero-Shot)
38
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#7CLIP-RN50 (Zero-Shot)
35
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code
#8CLIP-ViL (Zero-Shot)
15
Jaccard Index· 2022-07-25
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Code