Question Answering on MedMCQA

Metric: Dev Set (Acc-%) (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Dev Set (Acc-%)▼	Extra Data	Paper	Date↕	Code
1	Meditron-70B (CoT + SC)	66	No	MEDITRON-70B: Scaling Medical Pretraining for La...	2023-11-27	Code
2	Codex 5-shot CoT	0.597	No	Can large language models reason about medical q...	2022-07-17	Code
3	VOD (BioLinkBERT)	0.583	No	Variational Open-Domain Question Answering	2022-09-23	Code
4	Flan-PaLM (540B, SC)	0.576	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
5	Flan-PaLM (540B, Few-shot)	0.565	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
6	PaLM (540B, Few-shot)	0.545	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
7	Flan-PaLM (540B, CoT)	0.536	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
8	GAL 120B (zero-shot)	0.529	No	Galactica: A Large Language Model for Science	2022-11-16	Code
9	Flan-PaLM (62B, Few-shot)	0.462	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
10	PaLM (62B, Few-shot)	0.434	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
11	PubmedBERT(Gu et al., 2022)	0.4	No	MedMCQA : A Large-scale Multi-Subject Multi-Choi...	2022-03-27	Code
12	SciBERT (Beltagy et al., 2019)	0.39	No	MedMCQA : A Large-scale Multi-Subject Multi-Choi...	2022-03-27	Code
13	BioBERT (Lee et al.,2020)	0.38	No	MedMCQA : A Large-scale Multi-Subject Multi-Choi...	2022-03-27	Code
14	BERT (Devlin et al., 2019)-Base	0.35	No	MedMCQA : A Large-scale Multi-Subject Multi-Choi...	2022-03-27	Code
15	Flan-PaLM (8B, Few-shot)	0.345	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code
16	BLOOM (few-shot, k=5)	0.325	No	Galactica: A Large Language Model for Science	2022-11-16	Code
17	OPT (few-shot, k=5)	0.296	No	Galactica: A Large Language Model for Science	2022-11-16	Code
18	PaLM (8B, Few-shot)	0.267	No	Large Language Models Encode Clinical Knowledge	2022-12-26	Code

#1Meditron-70B (CoT + SC)SOTA
66
Dev Set (Acc-%)· 2023-11-27
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models Code
#2Codex 5-shot CoTSOTA
0.597
Dev Set (Acc-%)· 2022-07-17
Can large language models reason about medical questions?Code
#3VOD (BioLinkBERT)
0.583
Dev Set (Acc-%)· 2022-09-23
Variational Open-Domain Question Answering Code
#4Flan-PaLM (540B, SC)
0.576
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#5Flan-PaLM (540B, Few-shot)
0.565
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#6PaLM (540B, Few-shot)
0.545
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#7Flan-PaLM (540B, CoT)
0.536
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#8GAL 120B (zero-shot)
0.529
Dev Set (Acc-%)· 2022-11-16
Galactica: A Large Language Model for Science Code
#9Flan-PaLM (62B, Few-shot)
0.462
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#10PaLM (62B, Few-shot)
0.434
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#11PubmedBERT(Gu et al., 2022)SOTA
0.4
Dev Set (Acc-%)· 2022-03-27
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering Code
#12SciBERT (Beltagy et al., 2019)
0.39
Dev Set (Acc-%)· 2022-03-27
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering Code
#13BioBERT (Lee et al.,2020)
0.38
Dev Set (Acc-%)· 2022-03-27
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering Code
#14BERT (Devlin et al., 2019)-Base
0.35
Dev Set (Acc-%)· 2022-03-27
MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering Code
#15Flan-PaLM (8B, Few-shot)
0.345
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code
#16BLOOM (few-shot, k=5)
0.325
Dev Set (Acc-%)· 2022-11-16
Galactica: A Large Language Model for Science Code
#17OPT (few-shot, k=5)
0.296
Dev Set (Acc-%)· 2022-11-16
Galactica: A Large Language Model for Science Code
#18PaLM (8B, Few-shot)
0.267
Dev Set (Acc-%)· 2022-12-26
Large Language Models Encode Clinical Knowledge Code