detect hallucinations in langchain rag pipelines

okay so you're building a rag pipeline with langchain and your ai keeps making stuff up. been there. here's what actually works.

the problem: your bot sounds smart but lies

my customer support bot was telling people we had 24/7 support when we only work 9-5. it claimed we had "automatic refund processing" when everything's manual. subtle lies that sound totally reasonable.

the worst part? these aren't obvious hallucinations. they're plausible features we just don't have.

why it happens

your rag pipeline:

retrieves somewhat relevant docs
llm fills in gaps with "helpful" details
you get 70% truth, 30% fiction

detection method 1: see what's happening

first, add openllmetry to see everything:

from traceloop.sdk import Traceloop
Traceloop.init(app_name="my_rag_pipeline")

# your existing langchain code stays the same

now you can see exactly where the llm adds stuff not in your docs.

detection method 2: llm checking (75% accurate)

def detect_hallucination(context, response):
    prompt = f"""
    Context: {context}
    Response: {response}

    Does the response contain information not in the context? YES/NO only.
    """

    result = llm.invoke(prompt)
    return "yes" in result.lower()

detection method 3: pattern matching

these patterns almost always mean hallucination:

suspicious_patterns = [
    r'\d+\s*hours?',  # "48 hours"
    r'24/?7',  # "24/7 support"
    r'automatically',  # "automatically processed"
    r'real-time',  # usually a lie
]

the fix: better prompts

this cut my hallucinations by 60%:

ANTI_HALLUCINATION_PROMPT = """
Use ONLY information in the context. 
Do not add details not explicitly mentioned.
If information isn't available, say "I don't have that information."

Context: {context}
Question: {question}
"""

production setup that works

from traceloop.sdk import Traceloop
Traceloop.init(app_name="production_rag")

class ProductionRAG:
    def __init__(self):
        self.llm = OpenAI(temperature=0)
        self.prompt = ANTI_HALLUCINATION_PROMPT

    def query(self, question):
        result = self.qa_chain({"question": question})

        # check for hallucinations
        if detect_hallucination(context, result['answer']):
            # retry with stricter prompt
            strict_q = f"{question}\nOnly use exact information."
            result = self.qa_chain({"question": strict_q})

        return result['answer']

results

before: 30% of responses had hallucinations
after: <5% hallucination rate

cost: ~30% more for checking, worth it

quick wins

add openllmetry (2 lines of code)
use explicit anti-hallucination prompts
implement basic pattern detection
set temperature to 0
track what gets flagged

the scariest hallucinations are the plausible ones. "24/7 support" when you're 9-5. "automatic processing" when it's manual. with proper detection, you catch them before customers do.

tools that work

openllmetry: see everything
traceloop: track patterns
simple pattern matching: catches 90% of common lies

that's it. detect what your rag pipeline makes up, tell it to stop, verify it listened. your customers will thank you.

t12s @0xt12s