tStoryCloze

Topic Story Cloze

Introduced 2023-05-22

The tStoryCloze refers to the "Topic StoryCloze" benchmark, which is a spoken version of the StoryCloze textual benchmark. In the tStoryCloze, the goal is to evaluate continuation coherence given a spoken prompt. This version is designed to be easier compared to the fine-grained causal and temporal commonsense relations evaluated in the spoken benchmark called sStoryCloze. The tStoryCloze benchmark is used to assess the capabilities of SpeechLMs to maintain coherence in the spoken language given a prompt. This benchmark, along with the sStoryCloze, provides valuable insights into the performance of SpeechLMs in capturing different aspects of spoken content.