Skip to content

Tune hybrid lexical and vector search with cross-encoder reranking

Catch exact SKU numbers, error codes, and CLI flags that pure embeddings miss by fusing BM25 lexical search with neural reranking.

Copy the prompt and paste it into the AI assistant you already use.

Two-stage hybrid retrieval tuning checklist

  1. 01

    Rewrite multi-turn follow-ups into standalone queries

    Expand short follow-ups like 'What about on Android?' into 'How to configure push notifications on Android' before running search.

  2. 02

    Fuse BM25 sparse and dense vector candidates

    Combine Reciprocal Rank Fusion (RRF) across keyword and semantic indexes so exact acronyms, SKUs, and error codes always surface.

  3. 03

    Rerank top 25 candidates down to top 5

    Pass merged candidates through a fast cross-encoder reranker so only the highest-precision passages enter the LLM prompt.

  4. 04

    Track Recall@5 on real production queries

    Maintain a 100-query retrieval benchmark pairing real user phrasings with target chunk IDs to validate every index change objectively.

Questions and answers

Why do dense vector embeddings struggle with error codes and part numbers?

Dense embeddings compress text into broad conceptual meaning, often treating `ERR_4012` and `ERR_4019` or `Model-X2` and `Model-X3` as nearly identical.

How much latency does adding a cross-encoder reranker introduce?

Modern hosted or quantized rerankers score 20 to 30 short chunks in 80 to 160 milliseconds—often saving time by letting you pass fewer tokens to the LLM.

Should we use multi-query expansion on every single user message?

Save multi-query expansion for complex or low-score queries; simple single-rewrite + hybrid BM25/vector handles 90% of turns faster.

Learn more

Related guides and background articles for this task