Dear friends,
I think most people still underestimate the data center build-outs that will be required to meet demand. In 2024, I predicted that we would need more compute infrastructure for inference because of the large number of tokens consumed by agentic workflows. This remains true today. But many people still underestimate the compute infrastructure that will be necessary because agents, as well as people, increasingly use computers. We are in the early stage of agents causing demand for compute, storage, and networking to skyrocket.
Take web search. On a typical day, just one of my agents (paperreview.ai/, which reviews research papers) makes around 5-10K web search calls — dramatically more than I do manually. Similarly, when you use a consumer chatbot, a prompt can lead to multiple web search calls to produce a result. As agentic workflows grow, demand for web search will grow.
Or take analytics workloads. I've used coding agents to add analytics instrumentation to multiple applications. I further use agents to help me analyze this data and draw conclusions. Agents will store, retrieve, and analyze far more data than humans manually do. This increases demand for memory and for long-term storage.
Or for an industry-sector example, take banking. Before online banking, consumers could query their bank balances only by asking a bank teller or (later) using an ATM. Online banking caused the number of transactions to grow significantly, and many banks had to rearchitect their infrastructure to support this growth. As we move toward allowing agents to carry out financial transactions for us, the number of transactions will grow, and banks again will have to scale up their infrastructure to support this increased volume.
Agents will drive significant transaction volume across many applications, transaction types, and sectors of the economy. Many businesses will have to redesign their systems to handle this increased scale. And to make this possible, we will need significantly more data centers that provide compute, storage, and networking.
Many teams have been looking for ways to speed up LLM token throughput to speed up agentic workflows. However, we also have to speed up the tools that LLMs use. For some applications, tool calls, rather than tokens, are the bottleneck. For example, some of my data-analytics agents can quickly generate code to retrieve and process data, and the actual retrieval and processing then takes much longer. It often takes less time to generate a web search query than to carry out that web search and scrape the selected page(s).
I'm excited about the work that lies ahead to optimize the entire software stack for new agentic workloads. Building data centers — which are incredibly energy- and water-efficient for the work that they do — will be a key part of this effort. Unfortunately, the anti-data center movement, which has become a rallying point for a large number of people that distrust AI technology, do not like its impact, and want to slow it down, is making it much harder to build capacity. But I am optimistic that as the reality of AI's benefits become better understood, the tide will turn, and increasing data center capacity will lead to benefits for everyone.
Keep building!
Andrew
