Description-guided molecule generation on TOMG-Bench

Metric: wAcc (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	wAcc▼	Extra Data	Paper	Date↕	Code
1	Claude-3.5	35.92	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
2	Gemini-1.5-pro	34.8	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
3	GPT-4-turbo	34.23	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
4	GPT-4o	32.29	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
5	Claude-3	30.47	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
6	Llama-3.1-8B (OpenMolIns-large)	27.22	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
7	Galactica-125M (OpenMolIns-xlarge)	25.73	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
8	Llama3-70B-Instruct (INT4)	23.93	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
9	Galactica-125M (OpenMolIns-large)	23.42	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
10	Galactica-125M (OpenMolIns-medium)	19.89	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
11	GPT-3.5-turbo	18.58	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
12	Galactica-125M (OpenMolIns-small)	15.18	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
13	Llama3.1-8B-Instruct	14.09	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
14	Llama3-8B-Instruct	13.75	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
15	chatglm-9B	13.137	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
16	Galactica-125M (OpenMolIns-light)	13.136	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
17	Llama3.2-1B (OpenMolIns-large)	8.1	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
18	yi-1.5-9B	7.32	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
19	Mistral-7B-Instruct-v0.2	4.81	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
20	BioT5-base	4.21	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
21	MolT5-large	2.89	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
22	Llama-3.1-1B-Instruct	1.99	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
23	MolT5-base	1.3	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
24	MolT5-small	1.299	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code
25	Qwen2-7B-Instruct	0.15	No	TOMG-Bench: Evaluating LLMs on Text-based Open M...	2024-12-19	Code

#1Claude-3.5SOTA
35.92
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#2Gemini-1.5-pro
34.8
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#3GPT-4-turbo
34.23
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#4GPT-4o
32.29
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#5Claude-3
30.47
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#6Llama-3.1-8B (OpenMolIns-large)
27.22
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#7Galactica-125M (OpenMolIns-xlarge)
25.73
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#8Llama3-70B-Instruct (INT4)
23.93
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#9Galactica-125M (OpenMolIns-large)
23.42
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#10Galactica-125M (OpenMolIns-medium)
19.89
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#11GPT-3.5-turbo
18.58
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#12Galactica-125M (OpenMolIns-small)
15.18
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#13Llama3.1-8B-Instruct
14.09
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#14Llama3-8B-Instruct
13.75
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#15chatglm-9B
13.137
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#16Galactica-125M (OpenMolIns-light)
13.136
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#17Llama3.2-1B (OpenMolIns-large)
8.1
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#18yi-1.5-9B
7.32
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#19Mistral-7B-Instruct-v0.2
4.81
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#20BioT5-base
4.21
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#21MolT5-large
2.89
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#22Llama-3.1-1B-Instruct
1.99
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#23MolT5-base
1.3
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#24MolT5-small
1.299
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code
#25Qwen2-7B-Instruct
0.15
wAcc· 2024-12-19
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation Code