Retrieval RAG Evaluation Rita Fernandes Neves Senior Solution Architect - AI at NVIDIA Bilge Yücel DevRel Engineer Optimize RAG Applications with Document Reranking Using Haystack With NVIDIA NeMo Retriever March 20, 2025

Optimize RAG Applications with Document Reranking Using Haystack With NVIDIA NeMo Retriever In retrieval-augmented generation (RAG) applications, the quality of the retrieved documents plays a critical role in delivering accurate and meaningful responses. But what happens when embedding similarity is not enough to get an accurate ordering of the reference documents? This is where reranking comes into play. What’s Reranking? Reranking refers to assigning a relevance score to each document based on how well it matches the query. Reranking reorders the retrieved documents to ensure the most contextually relevant results are at the top. This is important because while the retrieval stage focuses on recall, considering relevance broadly, reranking “fine-tunes” the results for increased precision. Examples of Reranking Consider a query like, “What are the best practices for securing a REST API?” The retrieval model might return a ranked list with these documents: - REST API: a practical guide - Best REST API frameworks - Detailed steps on how to secure REST APIs - Public vs. private APIs: challenges and limitations - REST API architecture principles While all of these seem relevant to the topic of REST APIs, the document with specific security steps (document 3) should ideally be ranked first. Using purely embedding similarity, the document score may rely too much on common words - for instance, document 1 includes “REST API” and a similar word to “practice”, while document 2 also includes the word “best” from the query. The use of a reranker should lead to a better document scoring that overcomes these faults, leading to a better retrieval pipeline. Why Reranking is Crucial in RAG Systems Adding a reranking component to a RAG pipeline enhances both recall (retrieving relevant documents) and precision (selecting the most relevant ones). The reranker, typically using a fine-tuned LLM, reorders retrieved document chunks to ensure the most relevant ones appear at the top, making the retrieval process more accurate. By prioritizing the right documents, reranking increases the likelihood of providing the LLM with the best context, which improves the quality of generated responses. For example, in an application where the user seeks specific technical information, the reranking model ensures that highly relevant content appears first, preventing less helpful results from diluting the response quality. This is particularly important when the LLM providing the response has a limited context window or when we aim to optimize its inference process for speed and cost-efficiency. Reranking is especially valuable in hybrid retrieval setups, where chunks come from different datastores or from various retrieval methods (e.g., sparse, dense, or keyword-based). Each method may rank relevance differently, but reranking brings consistency regardless of the retrieval method. In hybrid setups, it ensures that the final set of documents provided to the LLM reflects the true semantic relevance to the query, rather than being dominated by a single retrieval method’s biases. Evaluation Metrics for Retrieval and Reranking Depending on the purpose, many metrics, such as semantic answer similarity or faithfulness, can be used to evaluate a RAG pipeline. When…