Settings

Theme

0.5B base model trained on 60B tokens rivals Qwen3.5-2B trained on 30T+ tokens

huggingface.co

12 points by AZDR · 2 comments

Reader

1 thread
cobychapple

it's 2.59B total with 553M active, so "0.5B vs 2B" more like 2.5B vs 2B? the card's own table has OLMo-2 1B (61.1) beating both.

  • AZDROP

    Fair points. What really matters IMO is total training compute and FLOPs/token. On that front:

    Nenya vs Qwen3.5 2B: Nenya has 71% fewer FLOPs/token and at least 99.8% less training compute (conservative assumption of only 10T tokens for Qwen, but the real number is likely 30T+).

    Nenya vs OLMo-2 1B: 57% fewer FLOPs/token (2.3×) and 99.4% less training compute (155×).

    Another way to look at it: at iso-compute (2e20 FLOPs), OLMo-1B (which is actually around 1.5B) scores 45.7 vs. Nenya at 58.3.

    Technical reports for both of these analyses: https://www.arsalanzafar.ai/nenya-technical-report

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection