The Agent Stack

01

Interface

The part of an agent you actually look at: where you type a request and watch the work come back. It handles presentation rather than execution, so the same agent can appear as a terminal, an editor panel, a native app, or a chat bot. The Agent Client Protocol calls this role the client.

Route

The harness services that request

Agents are more than just next token inference. They need to know information, and then take actions. The harness connects everything needed to do this: the context, the tools, and the LLM API.

02

Context

Everything the model gets to see before it answers. Most of it is pulled off your file system: AGENTS.md, skills, source files, and the output of earlier tool calls. This layer is where context comes from, not the context window itself; the harness decides what to load and in what order.

03

Harness

The code wrapped around the model that turns it into an agent. It runs one loop: read context, call the model, evaluate what came back, run the tools it asked for, then feed the results in and go again until the work is done. Claude Code, Codex, and Nori are each a harness.

04

Tools

What the agent can actually do to the world outside the model. Run a shell command, read and edit files, fetch a URL, call an MCP server. Each harness defines its own set and its own instructions for using them, so the same model behaves differently depending on which tools it was handed.

Return

The result comes back

The tool's result doesn't go to the model. It comes back to the harness. The harness reads the output, decides the next step, and only then calls the API. Every tool call round-trips through here.

05

API

The line where your control stops. Past this HTTP POST you cannot see or change how the request is served. Two things you can still set at the boundary: enterprise plan policy applied across a whole team, and an AI gateway, a stand-in API that meters spend per user, restricts which models are reachable, and issues one key for many providers.

06

Hyperscaler

The data center that runs the model. Your request lands in a provider region, is routed to a machine, and is served by two neighboring services: a cache and the inference service. The GPUs sit inside inference, and the model weights are loaded onto them.

07

Caching

Reusing the work of your last request instead of paying for it twice. Providers cache a prompt's prefix, so anything that changes early (a system prompt, a tool definition) invalidates everything after it. Those caches usually live five minutes to an hour, which is why harness choice and long gaps between turns show up directly on your bill.

08

Inference

The service that turns your tokens into the model's tokens. It reads the whole prompt in one pass, then generates one token at a time, batching your request alongside others to keep the hardware busy. Producing each new token normally means reading back over everything before it, so the service saves what that reading produced (two vectors, a key and a value, for every token so far) and reuses them instead. That store is the KV cache. It is why the second token is much cheaper than the first, and why a long conversation costs memory as well as time: the cache grows with every token in it.

09

GPU

The chip that does the arithmetic. A data center GPU packs a few hundred small processors, each holding tensor cores that multiply a tile of a matrix per instruction, alongside high-bandwidth memory. Generation speed is limited by how fast weights move out of that memory, not by raw math.

10

Model Weights

The learned numbers that are the model. A token enters at one end and passes through layer after layer of matrix multiplies until a single output token falls out the other. A dense model puts every token through every one of those numbers. A sparse one does not: in a mixture of experts, each layer is split into many separate blocks and a small router picks a few of them per token, leaving the rest untouched. This is why parameter count stopped predicting cost. A sparse model can carry a trillion parameters and still do the work of a far smaller one on any given token.

Complete system

One request crossed the application you shape, the provider boundary, and the infrastructure that evaluated the model.

req_7f3a · continuous route loop