Meta buys 600k H100s to train LLaMa3
twitter.comWrong (it's 350K according to Zuck on insta) and dupe of "Meta will have more than 350.000 Nvidia H100s this year"[0], or "Zuckerberg's Meta Is Spending Billions to Buy 350k Nvidia H100 GPUs"[1]
[0]: https://news.ycombinator.com/item?id=39044911 [1]: https://news.ycombinator.com/item?id=39046193
350k in new H100s and in total 600k in H100s equivalents.
I wonder what he means by "H100 equivalents" - are these GPUs built in-house or from competitors like AMD / Intel that give equivalent performance to Nvidia's H100s?
I wouldn't be surprised if the answer is "all of the above" and more. Not only is there limited availability of NVIDIA chips, it's also useful to tell Jensen that you actually could buy from somewhere else, even if you don't really want to.
At the end of the day, the bottleneck is TSMC, on which all GPUs are produced right now, even the ones from Intel.
I doubt they're paying retail prices like the tweet implies. Still can't be a small number if that is true.
If there is a discount, it will be tiny:
> “We’re so short on GPUs the less people use our products the better… We’d love it if they use it less because we don’t have enough GPUs” — Sam Altman
> Meta who has the 2nd most H100 GPUs in the world is increasingly leveraging GPU supply as a recruiting strategy.
https://www.datagravity.dev/p/2023-year-in-review-the-great-...
EXACTLY! what I was assuming.
Looks like it retails for $30K, I won't be surprised if the $20K is the bulk price they are paying, so the numbers could be correct.
Though as a wise man [1] once said: "The more you buy, the more you save".
[1] Jensen Huang, NVidia's CEO, says this quite often at conferences, its become a meme at this point, one example https://twitter.com/NVIDIAAI/status/917681161391841280
https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nomi...
lol, that's an investment!
$7 billion is a lot, but not comparable to the GDPs of countries which are in the trillions (unless you compare to the smallest of the long-tail).
Meta's annual revenues alone exceed $100B+ at this point, assuming these purchases are staggered over 2-3 years, it is basically adding a 2-3% overhead to their expenses on an annualized basis.
I can't wait to not be able to run the trained model.
I should have built a video card company based on VAX designs.
do it now.
How many "parameters" for the LLaMa3?
zuck won't puck.