Vulnerability Detection on VulScribeR

Metric: F1 Score (higher is better)

LeaderboardDataset

Loading chart...

Results

Submit a result

Hide extra data

Sort:

#	Model↕	F1 Score▼	Extra Data	Paper	Date↕	Code
1	Reveal Model - Tested on Reveal (Training on Devign + VulScribeR 20K + Extra Cleans)	26.18	Yes	VulScribeR: Exploring RAG-based Vulnerability Au...	2024-08-07	Code
2	Devign Model - Tested on Reveal (Training on Devign + VulScribeR 20K + Extra Cleans)	24.99	Yes	-	-	-
3	Reveal Model - Tested on Bigvul (Training on Devign + VulScribeR 20K + Extra Cleans)	18.98	Yes	-	-	-
4	Devign Model - Tested on Bigvul (Training on Devign + VulScribeR 20K + Extra Cleans)	18.51	Yes	-	-	-
5	LineVul - Tested on Reveal (Training on Devign + VulScribeR 20K + Extra Cleans)	17.38	Yes	-	-	-
6	LineVul - Tested on BigVul (Training on Devign + VulScribeR 20K+ Extra Cleans)	16.23	Yes	-	-	-

#1Reveal Model - Tested on Reveal (Training on Devign + VulScribeR 20K + Extra Cleans)SOTA
26.18
F1 Score· Extra Data· 2024-08-07
VulScribeR: Exploring RAG-based Vulnerability Augmentation with LLMs Code
#2Devign Model - Tested on Reveal (Training on Devign + VulScribeR 20K + Extra Cleans)
24.99
F1 Score· Extra Data
No paper
#3Reveal Model - Tested on Bigvul (Training on Devign + VulScribeR 20K + Extra Cleans)
18.98
F1 Score· Extra Data
No paper
#4Devign Model - Tested on Bigvul (Training on Devign + VulScribeR 20K + Extra Cleans)
18.51
F1 Score· Extra Data
No paper
#5LineVul - Tested on Reveal (Training on Devign + VulScribeR 20K + Extra Cleans)
17.38
F1 Score· Extra Data
No paper
#6LineVul - Tested on BigVul (Training on Devign + VulScribeR 20K+ Extra Cleans)
16.23
F1 Score· Extra Data
No paper