All projects
Case Study — 004

NotebookLM

Shipped, full RAG pipeline

An AI research assistant where every answer cites its source, down to the exact page, timestamp, or passage.

7
Source types
384-dim local
Embedding
6
Status stages
Hybrid + rerank
Retrieval
// Overview

Users create notebooks, isolated knowledge bases, then upload or link sources: PDFs, text, markdown, DOCX, websites, YouTube videos, VTT transcripts. Each source flows through an extract, chunk, embed, index pipeline with live status tracking.

Questions get streamed, natural-language answers carrying inline citations. Click a citation and the Source Viewer opens at the original page, timestamp, or highlighted passage. No claim without a path back to the source.

// Key Features
01

7 source types

PDF (pdf-parse), plain text, markdown, DOCX (mammoth), websites (Firecrawl to markdown), YouTube captions, and VTT transcripts.

02

Live indexing status

Sources move from pending to parsing to chunking to embedding to indexing to ready, with the UI polling every 2 seconds and one-click reindex on failure.

03

Hybrid retrieval

Qdrant semantic search (one collection per notebook) fused with Postgres tsvector keyword search, then reranked by Voyage rerank-2.5-lite.

04

Local embeddings

Supabase/bge-small-en runs locally via @huggingface/transformers. 384-dim ONNX inference, no embedding API bill.

05

Source Viewer

Citation chips resolve to the original material at the exact page, video timestamp, or highlighted passage.

06

Multi-notebook workspaces

Clerk-authenticated users keep isolated notebooks, each with its own sources, vector collection, and chat history.

// How It Works
  1. 01

    Upload or link a source. A BullMQ ingestion job lands on the queue (Upstash Redis)

  2. 02

    The worker extracts text, chunks it, embeds locally, and indexes into Qdrant, with status live at every stage

  3. 03

    Chunk text and tsvector vectors live in Neon Postgres (Drizzle) for keyword search

  4. 04

    On ask: query transform, then hybrid retrieve, then Voyage rerank, then DeepSeek streams the synthesized answer

  5. 05

    Every sentence carries a [[chunkId]] citation that the Source Viewer resolves back to the original material

// Stack
Next.js 16React 19Fastify 5ClerkNeon PostgresDrizzle ORMBullMQUpstash RedisQdrant CloudDeepSeekVoyage AIFirecrawlHF Transformers