Mega

Reported on 4 benchmarks across 2 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Machine TranslationonWMT2014 German-English
BLEU score· 2022-09-21
33.12
best: 35.15 (Bi-SimCut)
Mega: Moving Average Equipped Gated Attention arXiv:2209.10655
Machine TranslationonWMT2014 English-German
BLEU score· 2022-09-21
29.01
best: 35.14 (Transformer Cycle (Rev))
Mega: Moving Average Equipped Gated Attention arXiv:2209.10655
Machine TranslationonWMT2014 English-German
SacreBLEU· 2022-09-21
27.96
best: 33.8 (Noisy back-translation)
Mega: Moving Average Equipped Gated Attention arXiv:2209.10655

Medical1 result

Language ModellingonWikiText-103
Test perplexity· 2022-09-21
18.07
best: 2.4 (RETRO (7.5B))
Mega: Moving Average Equipped Gated Attention arXiv:2209.10655