DiscoSense: Commonsense Reasoning with Discourse Connectives

Prajjwal Bhargava, Vincent Ng

2022-10-22Sentence Completion

Abstract

We present DiscoSense, a benchmark for commonsense reasoning via understanding a wide variety of discourse connectives. We generate compelling distractors in DiscoSense using Conditional Adversarial Filtering, an extension of Adversarial Filtering that employs conditional generation. We show that state-of-the-art pre-trained language models struggle to perform well on DiscoSense, which makes this dataset ideal for evaluating next-generation commonsense reasoning systems.

Results

Task	Dataset	Metric	Value	Model
Sentence Completion	HellaSwag	Accuracy	91.5	ELECTRA-Large 335M (fine-tuned on DiscoSense and HellaSwag)
Sentence Completion	HellaSwag	Accuracy	86.9	ELECTRA-Large 335M (fine-tuned on HellaSwag)

Related Papers

Evaluating Gender Bias in Large Language Models2024-11-14 KatzBot: Revolutionizing Academic Chatbot for Enhanced Communication2024-10-21 BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs2024-07-14 Ranking LLMs by compression2024-06-20 Mixture-of-Subspaces in Low-Rank Adaptation2024-06-16 Enhancing Bangla Language Next Word Prediction and Sentence Completion through Extended RNN with Bi-LSTM Model On N-gram Language2024-05-03 MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts2024-04-22 Language Model Sentence Completion with a Parser-Driven Rhetorical Control Method2024-02-09