Ask HN: State of the GPU Rental Scene?
Who's got the best deals on VRAM by the hour these days? vast.ai was pretty bargain basement last time I used it. Anything peer to peer will be cheapest as you get to use non-datacenter GPUs and many sellers will be gamers happy to get free space heating from their sunk cost. Probably only scales so far before you need a Bedrock or similar and cough up.