Does a context gate for search agents actually work?

At Berlin Buzzwords 2026, Lester Solbakken gave a talk titled “Better retrieval makes agents worse” ¹Marginnote buzzwords1Lester Solbakken. “When better retrieval makes agents worse.” Berlin Buzzwords 2026. Talk recording. Lester builds Hornet.dev. ↩. These two slides matched my experience building search for agents.

Lester’s argument was that agentic retrieval is context admission control. An agent retrieves context to act. A false positive is not a wasted result on a page: it enters the context, gets re-read at every later step, and shapes the next action. Classic retrieval optimizes recall. A search tool inside an agent has to optimize precision.

The talk inspired me to build a context gate inside the search tool: a second model call that decides, per retrieved document, what enters the agent’s context. BrowseComp-Plus ²Marginnote browsecomp2Chen et al. 2025. “BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent.” ACL 2026. OpenAI’s BrowseComp queries (Wei et al. 2025, https://arxiv.org/abs/2504.12516) rehosted over a fixed 100K-document corpus with labeled gold and evidence documents, indexed with BM25 and Qwen3 embeddings. https://arxiv.org/abs/2508.06600 · dataset ↩ was the test-bed. It ships with labeled evidence documents for every query, which makes the idea measurable. The rest of this post measures it: six agent-and-gate configurations over the same 180 queries. Counting its own tokens, a cheap gate cuts input 1.4×, at accuracy indistinguishable from the ungated baseline. But on DeepSeek’s cache pricing the gate loses money, and it doubles per-query latency.

Why context accumulates

An agent with a search tool issues several search calls for a single question, and every result stays in the conversation until the final answer. By search 8, the agent is re-reading the distractors from search 1 on every step. Massive context windows make context stuffing the easy way out: leave everything in and trust the model to figure out the answer. It mostly works, and it pays for that in tokens and latency at every step.

Prior work has measured what it also costs in accuracy. The first-drop-of-ink paper Lester cites is worth a closer read.

Gao et al. ³Marginnote firstink3Gao, Chen, and Huang. 2026. “The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning.” The paper behind the talk’s ink slide: a small fraction of hard distractors causes most of the degradation, and filtering gains come mainly from context-length reduction rather than distractor removal. https://arxiv.org/abs/2605.10828 ↩ pin the damage on hard distractors, documents close enough to the topic to pass for evidence. The first such documents to enter the context do most of the harm.

In my ungated baseline run on BrowseComp-Plus, a single question costs the deepseek-v4-pro agent just over a million input tokens this way, 1,035K on average. The weaker flash agent averages 1.29M. Either agent writes roughly 1/100 of what it reads.

The context gate

The context gate is another agent/LLM call inside the search tool, one batched model call per search. Conditioned on the question, the gate assigns each retrieved document one of three admissions.

1. Full document. The complete document, truncated at a maximum token boundary.
2. Span. The gate returns only the snippets relevant to the question.
3. Withhold. The gate judges the document a distractor and returns only its title, with a note that it withheld the rest.

The gate is write-once. It decides what enters at retrieval time and never edits the conversation afterwards, because pruning earlier messages invalidates the prefix cache. What reaches the agent is only what the gate judged relevant at that moment.

Experiment setup

All the experiments are on BrowseComp-Plus: a fixed corpus of 100K passages, with queries labeled for gold documents (contain the answer) and evidence documents (needed to connect the hops).

The agent is DeepSeek (v4-flash or v4-pro depending on the arm) with a search and a get_document tool, capped at 30 iterations.

I am not building a better retriever here. The baseline retriever from the BrowseComp-Plus paper is hybrid BM25 + Qwen3-0.6B embeddings, at k=10. It surfaces a labeled document for about a quarter of the annotated queries, 180 in all. I call these the firing queries. The gate only decides what happens after retrieval surfaces something, so the experiments run on the queries where it has ground truth to act on.

uv run python -m search_agent.deepseek_client \
  --model deepseek-v4-pro --query runs/hybrid/newfiring182.tsv \
  --get-document --query-template QUERY_TEMPLATE \
  --searcher-type hybrid --bm25-index-path indexes/bm25 \
  --dense-index-path 'indexes/qwen3-embedding-0.6b/corpus.shard*_of_4.pkl' \
  --model-name Qwen/Qwen3-Embedding-0.6B --attn-implementation sdpa \
  --snippet-max-tokens 512 --k 10 --max-iterations 30 \
  --num-threads 16 --max_tokens 64000 \
  --gate model --gate-model deepseek-v4-flash \
  --gate-withhold soft --gate-spans-only \
  --output-dir runs/hybrid/newfiring182_proagent_flashgate_v3

The experiments

Before building a real gate, I tested the idea with an oracle that reads the answer key. The oracle admits only the labeled documents, with pre-annotated answer spans where they exist, and withholds everything else the search returns. With deepseek-v4-flash as the agent, the oracle gate scores 80% to the vanilla searcher’s 79%. And it uses 2.5× fewer tokens. Its score is a ceiling on withhold precision only, not on end-to-end accuracy. Incomplete annotations make the oracle withhold documents that are actually useful, so a real gate is not bound by it.

For judging the final answer, I re-use the BrowseComp-Plus judge mechanism, with deepseek-v4-flash as the judge model.

The deployable version has no answer key to look up. It reads each question–document pair and predicts the admission from the text alone.

Two of its design decisions came out of watching early runs fail.

1. Withholding must be soft.

The first version replaced withheld documents with a bare [withheld] marker. The agent treated every marker as a mystery box and called get_document on nearly all of them, pushing expansions and tokens above the ungated baseline. Showing the title fixed it. The agent has only one question about a withheld document, whether it is worth fetching, and the title answers it in a few tokens.

2. Spans beat full documents.

The best variant forbids admitting any document whole. Kept documents are compressed to the verbatim spans that connect them to the question. That bounds the context no matter how long the agent runs. One query in my set blows the model’s 1M-token context window when the gate admits whole evidence documents. Under spans-only it completes.

Results

Net of the tokens the gate itself reads, gating cuts the pro agent’s input 1.4× with no detectable accuracy change. The ungated pro agent scores 148/180 (82.2%). With deepseek-v4-pro as both agent and gate, it scores 147/180 (81.7%). One query apart is a tie at this sample size, where the error bars run about ±6 points. A band that wide cannot rule out a real drop of a few points either.

The agent’s own reads fall from 1,035K to 589K tokens per query, a 1.7× cut. Adding back what the gate reads nets out at the 1.4×. Swapping the pro gate for a flash gate keeps nearly all of that reduction and costs one more query, 146/180. Under the weaker flash agent the compression is deepest. Accuracy also drops six queries, from 143/180 (79.4%) to 137/180 (76.1%). The failure happens when one needed document is hidden and the agent wanders, issuing excess search calls.

What gating costs

The API costs come from each arm’s measured usage and DeepSeek’s price sheet ⁴Marginnote pricing4DeepSeek price sheet as of July 2026, per 1M tokens. v4-flash: $0.14 input (cache miss), $0.0028 input (cache hit), $0.28 output. v4-pro: $0.435 / $0.003625 / $0.87. Arm costs computed from each run’s measured cached/uncached/output token counts. Gate tokens are priced as cache misses, which makes the gate’s cost a slight upper bound. ↩.

API costs

The ungated baseline is the cheapest arm for its accuracy, because of context caching. The ungated agent re-reads a long, stable prefix on every step, and almost all of those reads are billed as cache hits. The gate reads each document once, in a fresh prompt, so its tokens are billed at cache-miss prices. Despite the token reduction, gating costs more per query.

89% of the baseline’s input tokens are cache hits. Cache-hit tokens are billed at 1/120th of the miss price. The 165M tokens of stale re-reading come to about $0.60. The tokens the gate removes were nearly free.

At cache-miss prices throughout, the ungated arm would cost $82.65 and the gated one $56.06. That is a 1.47× saving. A straight line between the measured run and that no-cache hypothetical puts the crossover near an 83% cache-hit rate. This run sat at 89%. On a well-cached agent with a 99% cache discount, stale context is almost free to re-read, and a gate has to justify itself some other way.

Latency

I did not instrument detailed timing during the runs. I did keep the agent traces, with timestamps, and each gate’s call counters. Every latency number below is a median, because API dropouts contaminate individual timings. On serial single-thread runs, the median query takes 83 seconds ungated and 165 seconds gated, twice as long.

The oracle arm applies the same gating semantics through a zero-cost label lookup, and it is faster than the baseline, a median of 66 seconds per query to the baseline’s 83. Admitting gold spans early ends queries in fewer searches.

The mechanism is one blocking model call per search, on the critical path by construction. The agent’s next turn conditions on the gated results, so there is no async escape. The gate memoizes decisions per document, but reformulated searches mostly surface new documents. In practice it fires 17–22 times per query, at 7–10 seconds a call. Those seconds are mostly decode. Spans-only means the gate writes the evidence out verbatim. That is about 800 completion tokens per call for the flash gate, 1,900 for pro. Spans-only bounded the context window, and spans-only decode is most of the added latency.

Cutting the overhead

Three levers:

1. Emit character offsets, not verbatim spans. The search tool already holds the document text. Let the gate return start:end and slice locally. That cuts the decode from ~800 tokens to under 100. Per call, an estimated 8 seconds drops to 1–2. This needs a gate model competent at emitting exact offsets. Smaller models struggle with it.
2. Gate documents in parallel. One batched call decodes every span serially, while k single-document calls decode only as long as the slowest one. Composes with the first lever.
3. A purpose-built small gate model. Distill the gate’s decisions into a 0.6–8B cross-encoder-style model served next to the agent: sub-second per search, no API round-trip. The agent×gate grid already showed gate capability is not the binding constraint. The flash gate ties the pro gate under a pro agent. Ding et al. see the same shape: robustness to retrieval noise rises with agent strength ⁵Marginnote robustrag5Ding et al. 2025. “On the Diminishing Returns of Complex Robust RAG Training.” SIGIR-AP 2025. The robustness benefit of defending an LLM against noisy retrieval shrinks substantially as model capacity grows. https://arxiv.org/abs/2502.11400 ↩.

Where the losses come from

I read the trajectory of every query a gated arm lost against the baseline, with the gate decisions alongside. (One direction only: the queries the gated arms won have not had the same close read.)

The failure mode specific to gating is trajectory steering. The gate withholds a document early, and the agent’s later query reformulations drift away from the gold documents. On one query the baseline retrieved all three gold documents. The gated agent issued 56 search calls across its 30 iterations and retrieved none of them. A per-search gate cannot know its decision will derail the search three steps later. This affects 8 of 180 queries, 4.4%.

I had been preparing the training side of this idea: distill the gate into a small open-weights model, then RL with trajectory-level reward to fix this steering. The census shelved the RL half. Steering affects 4.4% of queries, so at best RL recovers half an accuracy point. That gain sits entirely inside the ±6-point noise band. Distillation is still worth doing, but only for the 7–10 seconds per gate call.

The census also showed the largest failure bucket has nothing to do with gating. The agent holds the right documents, hits the 30-iteration cap without committing to an answer, and emits a raw tool-call fragment. On the oracle arm this accounts for about 9 of its 15 losses. For accuracy on this benchmark, termination handling is worth more than any gate improvement.

One benchmark, one retriever, one model family

Everything above is measured on one setup. BrowseComp-Plus queries are multi-hop questions over a fixed 100K-passage corpus, and the 180 firing queries are the subset where the stock hybrid retriever surfaces labeled evidence. At this sample size the error bars are about ±6 points, wide enough to swallow every accuracy difference between arms. The paired McNemar test in the results section says the same thing. Agents, gates, and judge are all DeepSeek v4 models. deepseek-v4-flash scored every arm, one trajectory per query. A judge from the same family as the system it grades can share that system’s blind spots, and one trajectory gives its quirks no chance to average out. A different agent family may react differently to gate-compressed context. I did not vary the gate prompt, so its sensitivity to wording is unmeasured. And nothing here says how the gate behaves on an unlabeled corpus, where no annotation ever checks its withhold decisions.

Takeaways

On this DeepSeek setup, the gate holds up as compression, 1.4× fewer input tokens with no training. The ink paper reaches the same verdict from a controlled direction. In its filtering experiments, the gain came from the shorter context rather than from which documents were removed ³. It loses on cost here because DeepSeek’s cache discount makes stale context nearly free to re-read. The dollar case needs a setting where caching is weak or absent. And as built, it costs about 2× wall-clock latency, most of which the levers from the latency section would remove. I have not built them yet.

When to use a gate

• Cache-hit rate around 83% or lower. Below that crossover the gate saves money as well as tokens. The 83% is interpolated from one measured run, so it is approximate. Multi-turn products that interleave user messages between searches, and serving stacks without prefix caching, sit well under it.
• The context window binds before the budget does. Spans-only admission caps what each search can add, so the context stays bounded however long the agent runs.
• A few extra seconds per search are acceptable. Each gate call blocks for 7–10 seconds as built. Character offsets and parallel calls remove most of that, but only after you build them.
• Never for accuracy. No configuration detectably beat the ungated baseline, and on this benchmark termination handling is worth more than any gate improvement.

Code, gate prompts, experiment logs, and the analysis behind every number in this post are in the BrowseComp-Plus fork, branch public/context-gating.