No, I get this; it is not a problem. It becomes one when they cannot pay back this financing, in the event that they cannot build a sustainable business, which they cannot, because the capital cycle leads to overinvestment, meaning the financiers cannot meet their returns.
Taking a multi-decade perspective, I wonder if our general analysis is focused too much on the initial wave of LLM tech and current gen GPUs.
Owning a massive data center connected to water and power and network that can be targeted or converted to developing needs seems like a decent problem to have for the big cloud companies. We have compute hungry companies and media, in addition to cryptocurrencies etc, and we’ll have more of them in 2045.
I don’t know if I’m underestimating how purpose-built these datacenters are, or overestimating the accountants in the corporate vehicles building them, but the broader situation doesn’t seem as fragile as 1929 or 2008 (even if the businesses are overvalued and LLMs fall totally out of fashion).
H100, almost a five year old GPU, costs more to buy used now than it was to buy brand new at release.
Of course, everything has a lifespan.
Consider a simple arithmetic problem, and this mania will start making sense.
An H100 costs approximately 30k. You can run a decent latest open model on it at 1000 tokens per second batched. Cost on open router is 4$ per 1m tokens.
That's about 120k revenue per year if there's demand. So far, there's unlimited demand.
You, as one person, can likely not make the logistics of this work. But this really works with the economies of scale.
Now, because of that everyone wants to buy GPUs and we don't have enough.
The math works much better with a newer GPU that produces more tokens per second and consumes less energy to do so, even if it costs double. So why would anyone buy an old one? Because demand is orders of magnitude more than supply.
https://s-1.vercel.app/posts/the-capital-cycle-theory/