Where are people finding GPU capacity?
Is it just me or is GPU capacity surprisingly hard to find right now , even in Spot pools? I’ve been trying to get H100s and the Spot pools I’ve checked are pretty much always full.
I ended up trying MI350/MI355s on DigitalOcean Spot and have been running them for about a week without getting interrupted. Curious if anyone else has tried them? How often are you actually getting reclaimed?
Also, where are you guys finding capacity these days?
It depends. GPU capacity is incredibly limited across the space right now.
Vast.ai is a good bet if you need something on demand and are willing to compromise on quality (frequently hosted in non-datacenter environments (non-ideal temps, network, etc)).
EC2 P5 instances are also semi-available (H200 though)
Vast.ai has some u could check out
Spot GPU capacity at DO has not disappointed me so far , best thing is the 2 hour reclaim window the fact that I can checkpoint while running our training workloads on the beefy Mi355x is awesome