Improving Throughput by Optimising KV Cache Efficiency for Agentic Workloads
j9s.io
1 thread
I really like this article, agent harness knows its own state: which tool call is running, whether a sub-agent is pending, when it is finished. The API throws that away because each request looks stateless. Interesting idea.