Multi-agent systems inherit every microservices problem.
Press enter or click to view image in full size
Pick any pain point people are hitting with multi-agent systems right now. Microservices teams were working through exactly these same problems circa 2010. It’s the same engineering rethinking, just with non-deterministic workers.
Looking for signals
During a recent AWS webinar, a poll on agent architecture said that:
- 10% of the audience already use some form of messaging/streaming for their agents
- 25% plan to within the next two months
- 55% said they “need to learn more”.
As CTO in the data streaming space, I know that enterprise agentic workflows have the same needs as any distributed application: traceability, resiliency, backpressure, scalability, replay. That’s what Kafka was built for.
Why? You wire two agents together with a direct call to start with. Easy. Then you add more because you are building more workflows. Soon enough you’re debugging communication and quality issues between an orchestrator and its sub-agents.
Agents aren’t a new category of system. They’re microservices with a brain, and they inherit every microservices problem: coupling, scaling, observability, debugging under non-determinism. The answer? It’s to put an event broker between them, often Kafka.
The rest of the post: why replacing direct calls between agents to be asynchronous, and where Kafka, MCP, and A2A each fit.
Three generations of LLM systems, stacked in three years
1/ The chatbot came first. One LLM (large language model), dedicated sessions, then some retrieval-augmented generation (RAG) on the side to help with context.
2/ Then the autonomous agent: an LLM plus tools, a filesystem, a sandbox, and a loop (REPL). You give it a goal, it decides what to do and has side-effects (writing files, calling APIs, executing programs). That’s what Claude Code or Codex are.
3/ Then multi-agent systems. Several agents collaborating on a task. A main agent dispatching work to specialized sub-agents. You now have a distributed system, and you have to treat it like one. This is what we find in enterprises.
Press enter or click to view image in full size
When multi-agent is legitimate (and when it isn’t)
Most problems don’t need multi-agent. A single agent with a good tool catalog usually beats a poorly orchestrated swarm. I built several orchestrators before it was a thing, all are deprecated now. Every AI vendor ships a capable agentic runtime of their own. That being said, it’s still useful for these reasons:
- Context isolation: a single agent’s context can bloat fast: many tools, a long system prompt, many messages, RAG chunks, and the model starts degrading on reasoning because of lack of attention. Splitting the work into sub-agents gives each one a clean, scoped context. The issue isn’t really the window size (today, we enjoy 1M of tokens), it’s the noise inside the window. Doubling the context doesn’t double the useful information, it dilutes the attention of the LLM (hence why it’s useful to often “/clear” and start fresh).
- Parallelization: when a task fans out into independent subtasks (deep research across multiple domains, diagnostic checks across multiple subsystems), running them sequentially is wasteful. Sub-agents with their own contexts go wide, and the orchestrator folds the results back.
- Specialization: once an agent has 30 tools to choose from, it might get confused about which one to call, may pick the wrong one, loop to find its way. Splitting those tools across specialized sub-agents with their own system prompt and mission works noticeably better.
Same rule applies to Kafka: it’s often added for the wrong reasons when a simple Postgres would be enough. Kafka is for scale and fan-out, not for low-throughput topologies. Avoid it as long as possible, until you genuinely need the async flow, the replay, and the consumer-group semantics, and you understand why. Otherwise you’re paying an ops tax (cluster, partitions, consumer-group semantics, monitoring, etc.) for benefits you don’t need.
The usual scenario
You wire the agents with direct HTTP calls. Agent A calls agent B, which calls tool C, which calls sub-agent D. HTTP, gRPC, doesn’t matter. You ship it, good.
You start to realize that you built something for the nominal use-case where everything is perfect and fast, but “failure modes” are invisible until they hit: backpressure, cascade timeouts, delays, retries, replay, and more:
- Connection complexity scales as N², and every new agent multiplies the integration surface. Temporal coupling means that if sub-agent B is slow, agent A is blocked, and if B is down, A fails.
- Direct calls can’t absorb spikes, so one burst of user messages and your orchestrator’s request queue fills up, might die and you lose what was in progress. A tool timing out cascades back through the call chain and dead agents block progress of work.
- Debugging is the worst part. You cannot replay a failed run. Good luck producing a minimal reproducer of an LLM-driven bug when the event that caused it is already gone (transient due to HTTP).
This is the same diagram microservices teams were drawing circa 2010. Most of them eventually put Kafka (or something close enough) in the middle for the same reasons. Direct synchronous calls were the wrong primitive fifteen years ago. They still are, and even more, due to LLM latency.
Press enter or click to view image in full size
MCP, A2A, and Kafka are not the same problem
MCP and A2A sit in a different layer than Kafka.
- MCP is tool calling. It standardizes how an agent talks to a tool or a resource: files, APIs, databases, whatever. It’s the contract between the agent and the outside world for a single action.
- A2A, Agent-to-Agent, is discovery and interop. Every agent publishes its card: here’s what I do, here’s my task format, here’s how to hand me a job. It solves the “how does agent A know agent B exists” problem.
- Kafka is the nervous system. It’s the layer where the messages flow at scale, with durability, with ordering, with replay. It’s what carries the MCP tool calls and the A2A handoffs across the runtime when the system is under load.
You want them all, let’s focus on Kafka, which is the base of the architecture.
Kafka pattern: Orchestrator-worker
Press enter or click to view image in full size
A microservice (an agent) publishes tasks it wants to a topic. A consumer group of sub-agents picks them up and processes them in parallel. Results come back on a response topic. The orchestrator consumes the responses and continues its reasoning.
Get Stéphane Derosiaux’s stories in your inbox
Join Medium for free to get updates from this writer.
Scaling out is mostly free: add more consumers, Kafka rebalances, throughput goes up. The orchestrator does not care whether a sub-agent is healthy, slow, or restarting. It publishes and moves on. No temporal coupling.
The response will eventually arrive and the orchestrator, who has a internal state, will correlate it using a conversation_id (probably as your Kafka partition key, to keep a single user’s traffic ordered on one partition).
Kafka gives full auditability for free
Every message in, every tool call, every sub-agent handoff, every response out, all of it published to topics. The conversation becomes an append-only log.
- Replay is where Kafka is extremely useful. You can replay the exact same event sequence against an application and watch the agent reason through it once again. It only works when the whole conversation exists as a durable, ordered log.
- Dead letter topics catch tool failures and agent timeouts. You alert when this happens and you replay them when the issue is fixed. It’s a standard pattern and it matters even more because workers are non-deterministic, so you’ll get failures that aren’t bugs in your code but in the model’s reasoning.
A real multi-agent architecture, end-to-end
Take this architecture: a digital-employee-experience agent that helps IT users resolve problems through chat and voice, deployed across millions of endpoints.
Press enter or click to view image in full size
The components:
- Ingestion. Messages arrive at a microservice API (queried from various places interacting with users) whose job is to land the message in a Kafka topic, the source of truth for “a user said something.”
- Main agent. A consumer group listens to this ingestion topic. This is the agentic loop: read message, reason, call tools, call sub-agents, publish requests to other topics to trigger more downstream agents, and eventually publish the response back. It runs as a graph execution (LangGraph-style), and the graph state is checkpointed to a database. That’s how you scale an agent loop to millions of concurrent users.
- Tool calling. Tools are async by design. The agents publish tool calls to a dedicated topic. Downstream services (action execution, workflow engine, data query layer) consume that topic, do their work (which can take seconds, minutes, or hours), and publish results back.
- Sub-agents. Specialized workers for troubleshooting Windows, troubleshooting macOS, and so on. The main agent publishes the task on a sub-agent topic. The sub-agent consumes it, runs its own agentic loop with its own isolated context, and publishes back a summary.
- Audit cockpit. This reads all topics involving agents and tooling, and stores them in a NoSQL store, runs summarization and intent classification on top.
Another big win is also organizational. Putting Kafka at the center delivers team independence. When the source of truth is a topic any team can subscribe to, new features become new consumer groups, not new endpoints on the agent service. That single property is what lets a small team ship on top of this without asking permission from the agent team every time.
The new problems Kafka introduces
Every Kafka aficionados will see it coming. Kafka solves the communication problems, but introduces other problems for agentic workflows:
- Out-of-order events. Three tools are called in parallel. Results come back in an unpredictable order. The agent has to handle the case where a result arrives for a step that’s no longer the current step. You need explicit logic in the main agent: wait for all expected responses, react on first-to-arrive, or reconcile late arrivals against the current graph state.
- At-least-once delivery/idempotence. Kafka’s default guarantee is at-least-once, except if you are using Kafka transactions (that’s quite rare). This means you may occasionally see the same message twice. The fix is that every message carries a unique ID, and every consumer should maintain a dedup cache (keyed on message ID, with a short TTL), to discard duplicates (already processed).
- Retries and timeouts. When a tool fails (web search timeout, knowledge base unavailable), the agent should retry a few times with backoff, then give up and move on. Agents that retry forever on a failing dependency cause cascading outages. Let the model reason about the failure and decide what to do next. Same pattern as circuit-breakers between microservices.
- Event conflicts. Multi-agent means multiple responses converging on the main agent, sometimes contradicting each other. Tool A says the device is fine, sub-agent B says it’s broken. The main agent needs a conflict resolution strategy, usually “prefer the more specific signal” or “re-query on disagreement.” That’s not a Kafka problem, strictly. But Kafka is what makes the conflict visible in the first place.
Invest in tracing from day one: ensure you have conversation/correlation IDs on every message, propagated across topics, and queryable end to end. You don’t get distributed tracing for free with Kafka. You have to instrument it.
Conclusion
Multi-agent is a distributed system. Most of the mistakes I see come from teams that don’t treat it that way.
Don’t go multi-agent unless the problem genuinely requires it. Context isolation, parallelization, and specialization are the three reasons I’d split a workload. Everything else is an over-engineered single agent. Same logic applies to Kafka: don’t add it until the debugging pain, fan-out, and concurrency occur.
The one thing that’s genuinely new is that your workers are now non-deterministic.