Coursify

Retrieval-Augmented Generation (RAG) — From Fundamentals to Production-Ready Agentic RAG Systems

Implementing Hybrid Retrieval with Re-Ranking

Combine dense retrieval, BM25, and cross-encoder re-ranking in the capstone. Implement a retrieval pipeline that balances speed and accuracy for production use.

Learning Goals

  • Implement hybrid retrieval with re-ranking
  • Build a production-grade retrieval pipeline

Implementing Hybrid Retrieval with Re-Ranking

A global tech support agent cannot rely on semantic "vibes" alone. If a user asks for a specific error code like ERR_9021, vector similarity might return general "error handling" documents. To solve this, we will implement a Hybrid Retrieval strategy that combines the semantic power of embeddings with the keyword precision of BM25. We will then pass the combined results through a Cohere Cross-Encoder to ensure the most accurate answer is always at Rank 1.

In this section, we will build the "Engine" of our capstone retriever.

Learning Goals

  • Fuse Dense and Sparse retrievers using Reciprocal Rank Fusion (RRF).
  • Integrate a Cross-Encoder for high-precision re-ranking.
  • Implement metadata pre-filtering to narrow the search space.

Core Concepts

1. The Power of Fusion

By combining OpenAI Embeddings (Dense) and BM25 (Sparse), we capture both the "Meaning" of a question and the "Exact Tokens" (IDs, product names).

2. Cross-Encoder Refinement

Bi-encoders (vectors) are fast but approximate. Cross-encoders look at the query and document together, which is much slower but far more accurate. We use this as a "Second Stage" to pick the Top 5 from an initial pool of 25.

3. The Retrieval Pipeline

Building the Capstone Retriever

  1. 1
    Step 1
    1from langchain_community.retrievers import BM25Retriever
2
3# all_chunks from the Ingestion step
4sparse_retriever = BM25Retriever.from_documents(all_chunks)
5sparse_retriever.k = 25
  2. 2
    Step 2
    1# vector_store from the Ingestion step
2dense_retriever = vector_store.as_retriever(search_kwargs={"k": 25})
  3. 3
    Step 3
    1from langchain.retrievers import EnsembleRetriever
2
3ensemble_retriever = EnsembleRetriever(
4    retrievers=[dense_retriever, sparse_retriever],
5    weights=[0.7, 0.3]
6)
  4. 4
    Step 4
    1from langchain_cohere import CohereRerank
2from langchain.retrievers import ContextualCompressionRetriever
3
4reranker = CohereRerank(model="rerank-english-v3.0", top_n=5)
5
6capstone_retriever = ContextualCompressionRetriever(
7    base_compressor=reranker, 
8    base_retriever=ensemble_retriever
9)

Common Mistakes

  • Ignoring Weights: If your system is mostly technical, a 0.5/0.5 weight might still favor the semantic vector too much. Try 0.3 Dense / 0.7 Sparse for part-number heavy datasets.
  • Latency Overload: Ensemble + Re-ranking adds ~1.5 seconds to each query. Ensure your UI shows a "Searching..." state to manage user expectations.

Recap

  • We implemented a two-stage retrieval architecture.
  • We used EnsembleRetriever for Hybrid search.
  • We added a Cross-Encoder to maximize the precision of the final context pool.

Knowledge Check

Question 1 of 3
Q1Single choice

Why do we perform re-ranking as a second stage instead of re-ranking the entire database?

Previous

Next