STAR Benchmark

Situated Reasoning

TextsVideosApache License 2.0Introduced 2021-12-06

How to capture the present knowledge from surrounding situations and perform reasoning accordingly is crucial and challenging for machine intelligence. STAR Benchmark is a novel benchmark for Situated Reasoning, which provides 60K challenging situated questions in four types of tasks, 140K situated hypergraphs, symbolic situation programs, and logic-grounded diagnosis for real-world video situations. (Data Download, STAR Leaderboard)

Benchmarks

Question Answering/Accuracy Video Question Answering/Average Accuracy Video Question Answering/Accuracy