That's roughly 11.66$ per hour per GPU, which is above the price that Google Cloud is reselling them at if you commit for 36 months: https://getdeploying.com/gpus/nvidia-b200

That's for standalone servers. xAI has something that nobody else has. The single largest interconnected cluster[1]. For inference it doesn't really matter, but for training that networking is crucial.

[1] Probably, there could be undisclosed clusters owned by other companies.

I would think that for training Google has optimized their entire setup for TPUs so I can’t see how this would be used for anything but inference

That's a good point. That makes it less likely that it's training, but who knows.

Plus electricity, labor, and maintenance?

Yeah I was wondering about this too. It seems like way too much per GPU considering the purchase price of a B200 was around $40k last time I checked. So if we naively ignore the price of electricity and maintanence, it would only take 5 months before renting is a worse deal than buying outright.

It’s a supply and demand thing. Google would definitely be buying from nvidia and setting up themselves if nvidia had the capacity.

SpaceX/xAi/musk are currently in a good market for “happening to own 100k cards we have nothing to do with”, and are exercising that control as hard as they can.