I Added Hybrid Search to Fix Retrieval. It Made It Worse.

I added a keyword branch to a vector retrieval system to fix one specific weakness, and the queries it was meant to fix got worse.

Vector search is strong on paraphrase and contextual similarity, and weak exactly where an actual keyword carries the meaning: a config parameter, an error code, a version string, a threshold value. For some queries (the ones including exact terms) the right document is not the most semantically similar one; it is the one that contains the string. Adding BM25, the standard keyword-relevance score, alongside the vector branch is the textbook move, and it is what I did.

What followed was not just slightly worse. On that same class of query, the one that was supposed to benefit from introducing BM25, the correct document stopped appearing at all. Not ranked low. Absent from the candidate set entirely.

Two things make this expensive. Nothing alerts. No error, no latency spike, no failed request. And users don’t report it. They conclude something. Nobody files a ticket about it.

Neither retrieval branch was broken. Both were behaving correctly. The failure was in the arithmetic that combined them, which forms the backbone of almost every hybrid search application.

A note on shape, since the rest of this depends on it. A hybrid retriever runs two searches side by side: one lexical, matching literal terms, and one vector, matching meaning. Their results get merged into a single shortlist, called the candidate set or the top-N. Only what survives onto that shortlist reaches anything downstream: a reranker, which re-scores it, and then the model that writes the answer. Everything below is about how that merge is done.

The queries that break vector search

Some questions aren’t conceptual. The user isn’t asking how something works. They’re asking what a specifically named thing is set to.

It’s easier to see with a domain-specific query. Let’s review one about Redis eviction policies:

maxmemory-policy default eviction

One fact is wanted by the user: which eviction policy Redis uses when none is configured. The answer is a single value, and it may live in exactly one place. Embedding similarity is the wrong tool here, because the right document isn’t the one that contains a lot of information about eviction policies. It is the one that contains the line stating the default.

This query class can be found in every vertical domain once you look: parameter names, version strings, error codes, field names, threshold values, medical concepts. And it is the query class where a purely semantic system comes up short. Adding a lexical branch so exact matches surface is the standard fix, and it’s correct in principle.

Two documents, one query

Take the Redis query above. Two documents compete.

Document A is a blog post, “Redis Eviction Policies Explained.” Around 700 words, mentioning ‘maxmemory-policy’ about fifteen times as it walks through all eight policies. It is indeed about the parameter, it just never says which one is the default.

maxmemory-policy noeviction

BM25 boosts Document A: the term appears often, and the document is short. Short and repetitive is the ideal BM25 profile. It scores Document B poorly for logical reasons: reference material states a fact once, and the containing document is long. The two scores are then added together.

The vector scores get it right: 0.85 against 0.60, although they are invisible in the sum.

The generated answer comes back fluent, explaining what each policy does, never mentioning which is the default. Nothing in it signals that the actual question went unanswered. That’s the danger of this failure.

The obvious implementation, and the trap inside it

In OpenSearch or Elasticsearch, the lowest-friction way to combine the two is a bool query with two should clauses:

{
  "query": {
    "bool": {
      "should": [
        { "match": { "content": "maxmemory-policy default eviction" } },
        { "knn": { "content_vector": { "vector": [ ... ], "k": 10 } } }
      ]
    }
  }
}

This is what most engineers can be expected to reach for first. It works, returns a merged result set with a single _score, and appears as if it’s doing the right thing.

However, in a bool query, a document matching multiple should clauses scores the sum of those clauses. The effective ranking function is BM25 + vector_score, a raw addition of two numbers on completely unrelated scales. The engine does not normalize across clauses. It adds whatever each one emits.

Those two numbers are not comparable, and not by a small margin:

• BM25 is unbounded. It rises the more often a term appears in a document, the rarer that term is across the corpus, and the shorter the document (term frequency, inverse document frequency, and length normalization). A short document repeating a distinctive term can emit a very high score, and there’s no upper bound.
• Vector similarity is bounded. Cosine-derived scores are within a narrow range, typically well under 1.

By adding them, you run a risk that the unbounded one could decide the ranking.

It compiles and returns results, doing the wrong arithmetic. Without a proper built-in evaluation pipeline, you may be unaware of the issue at all.

How to know if this is happening to you

Three checks, in order

First, are you on OpenSearch or Elasticsearch, and did you build hybrid retrieval as a bool query with two should clauses? If so you’re summing incomparable scores right now. That isn’t a hypothetical, it’s just what the query is doing.

Second, pull the per-clause score breakdown for twenty or so real queries. explain will give you this, or named queries if you’ve set them up. Put the two distributions next to each other. If your lexical scores range across something like 5 to 50 while your vector scores sit between 0.5 and 0.9, the vector branch isn’t really participating.

Third, put together a small set of queries where you know the answer lives in a reference-style document: a table, a config file, a spec sheet. Then check whether the correct document is in the candidate set at all, rather than checking its rank.

Two things that don’t save you

Tuning the boost doesn’t. Multiplying an already-incomparable score doesn’t make the scales comparable, it changes the slope of the domination. A boost multiplies a number that’s already on the wrong scale. It can shift which queries break, but no value makes the two comparable, because that isn’t what boosting does.

Reranking doesn’t either, and this is the part teams miss. Most debugging attention goes to the reranker and the generator, because that’s where the visible output is. But the damage here is done at candidate selection. Document B was displaced out of the top-N before the reranker ran.

What the principled fix looks like

The fix is to make the two branches comparable before combining them. Three options, in order of how much machinery they require.

• Use the purpose-built query instead of hand-rolling it. OpenSearch has a dedicated hybrid query type that normalizes each branch into a comparable range before combining. This is probably the single most useful thing to know here: the intuitive path adds raw scores, the purpose-built path normalizes, and most teams don’t find that out until something breaks.
• Normalize yourself, then weight. Rescale each branch into a common range and combine from there. On our two documents this puts the blog post and the reference at opposite ends of each scale, so your weights decide the outcome instead of the raw magnitudes.
• Route by query type. Classify the query and set the weights per class, so exact-term queries lean lexical and conceptual ones lean semantic. This addresses the trade-off a single global boost creates, at the cost of a classifier you then have to maintain.

Implementation notes, if you’re the one building it

• Min-max is only as stable as your extremes. A single BM25 outlier can stretch the range and flatten everything else. L2 is the steadier choice on noisy corpora.
• Geometric mean often behaves better than arithmetic, because it requires a document to score reasonably on both branches and pushes keyword-only matches toward zero.
• The query classifier can be a single LLM call, but that adds latency on every request. Budget for it before you commit to routing.

What I shipped

I capped how many documents BM25 could contribute to the merged set, setting that value to three.

That stops the overpowering BM25 from taking all the slots inside the context given to the LLM. Thus, a strong semantic match reaches the reranker instead of being crowded out before anything downstream can see it.

I’d call it a patch rather than a fix. Three is an arbitrary number, and some queries need more lexical results than that. More importantly, the two scores are still on completely different scales.

If you’re buying rather than building: “we use hybrid search” answers nothing. Ask a vendor how they combine the two scores, and whether they can show retrieval quality broken out by query type. A team that has thought about this will answer immediately. A team that hasn’t will start talking about what they’re doing.

What generalizes

The example here is Redis, but the underlying problem isn’t specific to it.

Any corpus that contains both explanatory writing and reference material has this fault line running through it. Explanations repeat terms, references state them once, BM25 prefers the explanation, and the reference is usually the thing that answers the question. That describes developer documentation, legal text, financial filings, clinical references and most internal wikis, which is to say most places people are pointing retrieval systems.

A few things I’d take away from it:

• Adding hybrid search without thinking about fusion can make retrieval worse. Going hybrid isn’t automatically an improvement.
• Check candidate selection before you blame the reranker or the model. The failure is often upstream of where everyone is looking.
• Boosting amplifies, it doesn’t normalize. If the search branches that you combine aren’t comparable, tuning the boost just relocates the problem.
• Evaluate by query class rather than in aggregate. This failure was specific to one class, and an average hid it completely, which is exactly what averages do.

If you’re running hybrid retrieval in production I’d be curious to compare notes, particularly if you’ve approached fusion differently and it’s held up.