o1-preview

Reported on 3 benchmarks across 2 tasks · 1 paper · 1 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing2 results

Code GenerationonWebApp1K-React
pass@1· 2024-09-19
0.952
SOTA
A Case Study of Web App Coding with OpenAI Reasoning Models arXiv:2409.13773
Code GenerationonWebApp1k-Duo-React
pass@1· 2024-09-19
0.652
best: 0.679 (claude-3-5-sonnet)
A Case Study of Web App Coding with OpenAI Reasoning Models arXiv:2409.13773

Knowledge Base1 result

Mathematical ReasoningonFrontierMath
Accuracy
0.01
best: 0.252 (o3)