Traditional Retrieval-Augmented Generation (RAG) systems often assume that retrieved documents are always relevant to the user's query. However, in real-world scenarios, retrievers can fail due to ambiguous queries, outdated vector stores, or poor embedding alignment. When a RAG system generates an answer based on irrelevant context, the result is often a hallucination. Corrective RAG (CRAG) addresses this by introducing an automated assessment layer that evaluates the quality of retrieved chunks and triggers fallback mechanisms, such as web searches, when necessary.
The Architecture of CRAG
The core philosophy behind CRAG is "assess, then act." Instead of blindly passing retrieved documents to the Large Language Model (LLM), CRAG uses a lightweight classifier or the LLM itself to judge the relevance of the top-k results. This judgment typically categorizes the retrieval into three states:
- Correct: The retrieved documents contain sufficient information to answer the query. The system proceeds with standard RAG.
- Ambiguous: The documents might be helpful but lack completeness. The system may refine the search or augment the prompt.
- Incorrect: The documents are irrelevant. The system discards the local context and triggers a web search to find fresh, reliable information.
Implementing the Relevance Assessment
While dedicated classification models can be used, a common and effective approach for CRAG is using the LLM to perform zero-shot or few-shot classification of the retrieval quality. Below is a Python example using LangChain to demonstrate how you might structure this logic.
from langchain_community.llms import OpenAI
from langchain.prompts import ChatPromptTemplate
def assess_relevance(query: str, context: str, llm: OpenAI) -> str:
"""
Uses an LLM to assess if the retrieved context is relevant to the query.
Returns 'Correct', 'Ambiguous', or 'Incorrect'.
"""
prompt_template = ChatPromptTemplate.from_template(
"""
You are a relevance classifier. Analyze the following query and retrieved context.
Query: {query}
Context: {context}
Determine the relevance of the context to the query.
- Correct: The context directly answers the query.
- Ambiguous: The context is partially relevant but missing key details.
- Incorrect: The context is completely unrelated.
Respond with only one word: Correct, Ambiguous, or Incorrect.
"""
)
chain = prompt_template | llm
return chain.invoke({"query": query, "context": context}).content.strip()
# Example usage logic
query = "What is the latest status of the Paris Agreement?"
retrieved_context = "The Earth is the third planet from the Sun..."
relevance_score = assess_relevance(query, retrieved_context, llm)
if relevance_score == "Incorrect":
# Trigger Web Search Fallback
print("Retrieval failed. Triggering web search...")
# web_search_results = search_engine(query)
# final_context = web_search_results
elif relevance_score == "Ambiguous":
print("Retrieval ambiguous. Augmenting with additional sources...")
else:
print("Retrieval successful. Using local context.")
The Web Search Fallback Mechanism
When the assessment yields an "Incorrect" result, CRAG shifts the burden of finding information from the static vector database to dynamic web search APIs. This ensures that the LLM has access to up-to-date and factually accurate data, significantly reducing hallucination rates. It is crucial to implement rate limiting and caching for these web searches to manage costs and API quotas effectively.
Conclusion
Corrective RAG represents a significant step forward in building robust LLM applications. By not trusting the retrieval process blindly, developers can create systems that self-correct and adapt to complex user queries. Implementing CRAG requires additional compute for the assessment step, but the trade-off is vastly improved answer accuracy and reliability, making it an essential pattern for production-grade RAG pipelines.