Auto Debugging on Big-bench Lite
Metric: Exact string match (higher is better)
Metric: Exact string match (higher is better)
| # | Model↕ | Exact string match▼ | Extra Data | Paper | Date↕ | Code |
|---|---|---|---|---|---|---|
| 1 | PaLM 62B (few-shot, k=5) | 38.2 | No | PaLM: Scaling Language Modeling with Pathways | 2022-04-05 | Code |
| 2 | PaLM 540B (few-shot, k=5) | 38.2 | No | PaLM: Scaling Language Modeling with Pathways | 2022-04-05 | Code |
| 3 | PaLM 8B (few-shot, k=5) | 14.7 | No | PaLM: Scaling Language Modeling with Pathways | 2022-04-05 | Code |