MODULE 5  ·  DAY 2

Docs Assistant — Naive RAG

Grounding an LLM in your own documents

Question about YOUR docs Retrieved context Grounded answer find generate Retrieval-Augmented Generation — no fine-tuning, just wiring

Gourav Shah  ·  School of DevOps & AI  ·  Hands-on

M5·01

What you'll learn

The anatomy of a real GenAI app — and how to wire it in containers.

12 34 5 Name the four parts of every GenAI application Understand a vector DB as search by meaning, not by title Walk the naive-RAG pipeline: ingest → chunk → embed → retrieve → generate Wire it in Compose — ChromaDB + app talking to native Ollama See where naive RAG breaks — setting up Module 6
M5·02

A model alone can't answer about YOUR docs

It answers confidently from training data that predates your runbooks.

Raw LLM — ungrounded "How do I restart the payments service?" plausible-sounding but WRONG command RAG — grounded same question + the actual runbook kubectl rollout restart deploy/payments — correct add docs
M5·03

Anatomy of a GenAI application

Four parts, wired together. You already built the first two in M2 and M3.

Application Streamlit UI · orchestrate LLM endpoint qwen2.5:1.5b · Ollama Embedding model nomic-embed-text · 768-dim Vector database ChromaDB · search by meaning already have it (M2/M3) compose grows by 2 services
M5·04

A vector DB shelves by meaning, not title

A filing cabinet needs the exact name. A librarian understands what you mean.

Filing cabinet (by title) "Payments Runbook" "SRE ops, tier 1" "Deploy notes" wrong words → nothing found Librarian (by meaning) ? question → coordinates → nearest shelf
M5·05

Embeddings: text becomes coordinates

An embedding model places each passage as coordinates in meaning-space.

"restart payments" "bounce the service" "the moon is round" Embedding model 768-dim space near far
M5·06

The naive-RAG pipeline

Ingest once. Then answer every question with the same embedding step.

INGEST · run once Load doc Chunk Embed Store → ChromaDB QUERY · every question Embed query Retrieve top-3 Augment prompt Generate same embedding model
M5·07

Where the pieces run

Models stay native for Metal; the vector store and app are containers.

Native Ollama :11434 qwen2.5:1.5b — generate nomic-embed-text Docker containers ChromaDB :8000 Streamlit app host.docker .internal
M5·08

ChromaDB — the lightest vector store

Zero-config and under 2 GB — the right default for a 16 GB laptop.

ChromaDB — default zero-config · one service Python-native · <10 lines persistent · ≤ 2 GB total pinned to 0.5.20 scale up when you outgrow it Qdrant millions of vectors · filtering pgvector already on Postgres? add a column
M5·09

Wired via environment variables

Hand-author the compose service by service — endpoints are just env vars.

compose.yaml model → Ollama OLLAMA_URL=host.docker.internal embeddings EMBED_MODEL=nomic-embed-text ChromaDB CHROMA_HOST=chromadb:8000 app (Streamlit) reads all of the above one block at a time portable across runtimes
M5·10

Learning Mode — watch the pipeline run

The app surfaces each step live — the invisible parts of RAG become visible.

12 34 Query embedding Similarity search Retrieved context LLM generation 768-dim vector + ms taken searched N, got top-3 the exact runbook lines gen time + which model
M5·11

Where naive RAG breaks down

It works when the question matches the wording. It fails in predictable ways.

Query mismatch — phrasing differs, misses the chunk Wrong chunk boundary — a 500-char split cuts a procedure Single-pass retrieval — one miss, no retry No query rewriting — jargon and typos hurt scores Stale index — docs changed, index didn't Module 6 Agentic RAG fixes every row
M5·12

TO THE LAB

Retrieve, then generate — grounded.

Hand-author the compose, ingest Acme's runbooks, watch Learning Mode explain.

"restart payments?" the exact runbook chunk kubectl rollout restart deploy retrieve generate

Next: Lab — build the Docs Assistant.  ·  Gourav Shah · School of DevOps & AI

M5·13