24. Estimating LLM demand in a YC batch
2023-04-11
Another YC batch has recently come and gone. Some have reported “about half” of the companies in the batch are commercializing AI in some way. [1]
This got me wondering: What might a given YC batch spend on LLMs?
Here’s a quick back-of-the-envelope calculation using very rough estimates and assuming relatively straightforward LLM usage:
130 companies [half of the batch]
x $8 in LLM API usage costs per user per month [based on usage statistics in a product I built on gpt-4 and embeddings]
x 500 users per month [for a conservative average estimate across the batch]
x 12 months [to annualize the spend]
This comes out to about $6 million in inference costs alone in the first year of the batch. This isn’t a lot, but the user base — fast-growing startups — makes it an interesting market.
On the one hand, many of these companies may wind down, transition usage to a cheaper option rather than an OpenAI or Anthropic model, potentially even build their own model, or pivot away from using AI-driven features altogether.
On the other hand, in the same way YC only needs one breakout company to return the batch, one or two users may grow LLM usage many orders of magnitude in the coming quarters and years.
YC companies like Stripe and Brex have nailed this YC playbook in the last few years. Going forward LLMs feel like an easy addition to that startup stack.