🚀 The Problem with "Chatting" with PDFs

We've all built the standard RAG app:

1. Upload a PDF ✅
2. Ask a question ✅
3. Get an answer ✅

It's useful, but it's passive.

Real learning isn't passive. A real teacher doesn't just answer; they quiz you, they track your progress, and they remind you when you're about to forget something.

I built Echo-Learn to solve this.

Echo-Learn is an open-source Agentic RAG Engine designed to turn any document—whether it's a coding manual, a medical textbook, or legal guidelines—into an active, voice-enabled study partner.

Here's the deep dive into building an AI that "learns how you learn."

🧠 The Core Concept: "Agentic RAG" + "Biological Memory"

Echo-Learn isn't just a wrapper around an LLM. It combines two powerful concepts:

🛠️ The Architecture & Code

The stack is built on:

• 🧠 Google Gemini 3 Flash → Reasoning
• 💾 Upstash Redis → Memory & State
• 📐 Upstash Vector → Hybrid Search (Vectors + BM25)
• 👁️ Mistral OCR → Document Understanding
• 🔊 ElevenLabs → Real-time Voice Streaming

Let's look at the code.

1️⃣ The "Echo" Mechanism: Coding the Forgetting Curve

The heart of Echo-Learn is its ability to track Mastery. We don't just store what you know; we store how well you know it and when you last recalled it.

We implemented a decay algorithm directly in our storage layer:

📁 packages/storage/src/redis/mastery.ts

const DECAY_RATE = 0.1; // λ - controls how fast you forget

function calculateEffectiveMastery(
  storedMastery: number,
  lastInteractionDate: Date
): number {
  const daysSince = daysBetween(lastInteractionDate, new Date());

  // Exponential decay: Mastery drops over time without review
  // Formula: M = S × e^(-λt)
  const decayFactor = Math.exp(-DECAY_RATE * daysSince);

  return storedMastery * decayFactor;
}

🎯 How It Works In Practice:

Day 0: You learn "React Hooks"      → Mastery = 100%
Day 5: You haven't reviewed it      → Mastery decays to ~60%
       ↓
       Echo-Learn: "Quick quiz on useEffect before we continue?"

The AI doesn't wait for you to ask—it proactively teaches.

2️⃣ The Brain: Mode-Aware Agentic Strategy

A study partner needs to know when to lecture and when to listen. We built a Mode-Aware Strategy using Vercel AI SDK's ToolLoopAgent.

📁 packages/agentic/src/strategies.ts

export async function executeUnifiedAgenticStrategy(
  query: string,
  options: ModeAwareQueryOptions
) {
  // The AI decides: Are we chatting, or are we LEARNING?
  const mode = options.mode || "learn";

  // Different modes get different prompts & tools
  const systemPrompt = buildModeSystemPrompt(mode, modeResult, userProfile);

  const agent = new ToolLoopAgent({
    model: google("gemini-3-flash-preview"),
    tools: getToolsForMode(allTools, mode),

    // Dynamic tool selection per step
    prepareStep: ({ stepNumber }) => {
      if (stepNumber === 0) return { toolChoice: "required" }; // Force RAG search
      return { toolChoice: "auto" }; // Then let AI decide
    },
  });
}

💬 The Personality Switch:

3️⃣ The Eyes: Semantic Chunking for Complex Docs

Textbooks have structure. If you just chop them into arbitrary 500-char blocks, you lose the lesson.

We use Semantic Chunking that detects topic shifts:

📁 packages/ingest/src/chunker/semantic-chunker.ts

function findTopicBreakPoints(
  sentences: SentenceInfo[], 
  threshold: number
): number[] {
  const breakPoints: number[] = [];

  for (let i = 2; i < sentences.length - 1; i++) {
    // Compare text BEFORE and AFTER this point
    const prevContext = getCombinedText(sentences, i - 2, i);
    const nextContext = getCombinedText(sentences, i, i + 2);

    const similarity = calculateTextSimilarity(prevContext, nextContext);

    // Topic changed? Start a new chunk here.
    if (similarity < threshold) {
      breakPoints.push(i);
    }
  }

  return breakPoints;
}

This ensures that a "Chapter on Authentication" stays together, not split across 3 random chunks.

4️⃣ The Knowledge: Hybrid Search with Upstash Vector

We don't just use semantic similarity. We use Hybrid Search (Vectors + BM25 Keywords).

📁 packages/storage/src/vector/client.ts

export async function searchWithEmbedding(
  query: string,
  options: SearchWithEmbeddingOptions = {}
): Promise<Array<VectorSearchResult>> {
  const { topK = 10, fusionAlgorithm = "RRF" } = options;

  // Upstash handles both dense (semantic) and sparse (BM25) search
  const results = await vectorIndex.query({
    data: query, // Text → Upstash auto-generates embedding
    topK,
    fusionAlgorithm: "RRF", // Reciprocal Rank Fusion
  });

  return results;
}

Why Hybrid Matters:

💡 Unlimited Use Cases: What Can YOU Build?

Echo-Learn is a platform, not just a single app. Swap the knowledge source, and you get a completely different product:

🎓 1. The Ultimate Study Guide

Input: University textbooks (Biology, History, Physics)

Experience: Upload a chapter → AI quizzes you → Tracks which formulas you keep forgetting → Creates personalized study schedule for finals week

🏢 2. Corporate "Brain" & Onboarding

Input: Employee Handbooks, Compliance PDFs, Technical Docs

Experience: New hire talks to Echo-Learn instead of reading 50 pages

• AI: "You read the security policy last week. Do you remember the phishing protocol?"
• User: "Uh..."
• AI: "Let's review that section."

⚕️ 3. Medical & Legal Certification Prep

Input: DSM-5, Legal Codes, Case Law

Experience: Professionals studying for Boards or Bar exams. The "Smart Memory" ensures high-stakes info is retained, not just skimmed.

🛠️ 4. Technical DIY Assistant

Input: Car repair manuals, Appliance guides

Experience: "I'm under the car—what's the torque spec for this bolt?" → AI retrieves the exact table row and reads it aloud.

📊 The Tech Stack At a Glance

🔗 Open Source & Ready to Fork

I believe Personalized Education is the killer use case for AI. Echo-Learn is the foundation.

It's fully open source. Fork it, plug in your own documents, and you have a custom AI Tutor in minutes.

⭐ Show Some Love

If you found this useful, please:

1. Star the repo → It helps others discover Echo-Learn
2. Fork & build → I'd love to see what you create
3. Drop a comment → What would YOU use this for?

Let's build the future of learning, one echo at a time. 🔊

Built with ❤️ using Gemini 3, Upstash, Mistral AI, and ElevenLabs