Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

RAFT

Realworld Annotated Few-shot Tasks

Introduced 2021-09-28

The RAFT benchmark (Realworld Annotated Few-shot Tasks) focuses on naturally occurring tasks and uses an evaluation setup that mirrors deployment.

RAFT is a few-shot classification benchmark that tests language models:

across multiple domains (lit reviews, medical data, tweets, customer interaction, etc.)
on economically valuable classification tasks (someone inherently cares about the task)
with evaluation that mirrors deployment (50 labeled examples per task, info retrieval allowed, hidden test set)

Description from: https://raft.elicit.org/

Image source: https://raft.elicit.org/

Benchmarks

Classification/Avg Classification/ADE Classification/B77 Classification/NIS Classification/OSE Classification/ Over Classification/SOT Classification/SRI Classification/TAI Classification/ToS Classification/TEH Classification/TC Few-Shot Text Classification/Avg Few-Shot Text Classification/ADE Few-Shot Text Classification/B77 Few-Shot Text Classification/NIS Few-Shot Text Classification/OSE Few-Shot Text Classification/ Over Few-Shot Text Classification/SOT Few-Shot Text Classification/SRI Few-Shot Text Classification/TAI Few-Shot Text Classification/ToS Few-Shot Text Classification/TEH Few-Shot Text Classification/TC Text Classification/Avg Text Classification/ADE Text Classification/B77 Text Classification/NIS Text Classification/OSE Text Classification/ Over Text Classification/SOT Text Classification/SRI Text Classification/TAI Text Classification/ToS Text Classification/TEH Text Classification/TC

Statistics

Papers: 18
Benchmarks: 36

Links

Tasks

Classification Few-Shot Text Classification Text Classification