MODULE 5  ·  DAY 2

Docs Assistant, Naive RAG

Ground an LLM in your own documents

Question about YOUR docs Retrieved context Grounded answer find generate Retrieval-Augmented Generation, no fine-tuning, just wiring

Gourav Shah  ·  School of DevOps & AI  ·  Hands-on

M5·01

What you'll learn

Five things before you open the lab.

12 34 5 Name the four parts of every GenAI application Understand a vector DB as search by meaning, not by title Walk the naive-RAG pipeline: ingest → chunk → embed → retrieve → generate Wire it in Compose, ChromaDB and app talking to native Ollama See where naive RAG breaks, setting up Module 6
M5·02

1 · The problem: ungrounded answers

M5·03

A model alone can't answer about YOUR docs

Confident, but not about your systems.

Raw LLM, ungrounded "How do I restart the payments service?" plausible-sounding but WRONG command RAG, grounded same question + the actual runbook kubectl rollout restart deploy/payments, correct add docs
M5·04

2 · Anatomy of a GenAI application

M5·05

Four parts make every GenAI app

You already built two of them in M2/M3.

Application Streamlit UI · orchestrate LLM endpoint qwen2.5:1.5b · Ollama Embedding model nomic-embed-text · 768-dim Vector database ChromaDB · search by meaning already have it (M2/M3) compose grows by 2 services
M5·06

3 · The librarian analogy

M5·07

A vector DB shelves by meaning, not title

A filing cabinet needs the exact name.

Filing cabinet (by title) "Payments Runbook" "SRE ops, tier 1" "Deploy notes" wrong words → nothing found Librarian (by meaning) ? question → coordinates → nearest shelf
M5·08

Embeddings: text becomes coordinates

Similar text lands close together.

"restart payments" "bounce the service" "the moon is round" Embedding model 768-dim space near far
M5·09

4 · The naive-RAG pipeline

M5·10

Ingest once, then answer every question

Same embedding step, both phases.

INGEST · run once Load doc Chunk Embed Store → ChromaDB QUERY · every question Embed query Retrieve top-3 Augment prompt Generate same embedding model
M5·11

Where the pieces run

Models native for Metal, everything else in containers.

Native Ollama :11434 qwen2.5:1.5b, generate nomic-embed-text Docker containers ChromaDB :8000 Streamlit app host.docker .internal
M5·12

5 · ChromaDB: the lightest vector store

M5·13

ChromaDB, the lightest vector store

Zero-config, under 2 GB, the default here.

ChromaDB, default zero-config · one service Python-native · <10 lines persistent · ≤ 2 GB total pinned to 0.5.20 scale up when you outgrow it Qdrant millions of vectors · filtering pgvector already on Postgres? add a column
M5·14

Wired via environment variables

One service block at a time.

compose.yaml model → Ollama OLLAMA_URL=host.docker.internal embeddings EMBED_MODEL=nomic-embed-text ChromaDB CHROMA_HOST=chromadb:8000 app (Streamlit) reads all of the above one block at a time portable across runtimes
M5·15

6 · Learning Mode: watching the pipeline run

M5·16

Learning Mode, watch the pipeline run

Watch each step happen, live.

12 34 Query embedding Similarity search Retrieved context LLM generation 768-dim vector + ms taken searched N, got top-3 the exact runbook lines gen time + which model
M5·17

7 · Where naive RAG breaks

M5·18

Where naive RAG breaks down

It fails in predictable, fixable ways.

Query mismatch: phrasing differs, misses the chunk Wrong chunk boundary: a 500-char split cuts a procedure Single-pass retrieval: one miss, no retry No query rewriting: jargon and typos hurt scores Stale index: docs changed, index didn't Module 6 Agentic RAG fixes every row
M5·19

TO THE LAB

Retrieve, then generate. Stay grounded.

Build it, then watch it ground an answer.

"restart payments?" the exact runbook chunk kubectl rollout restart deploy retrieve generate

Next: Lab — build the Docs Assistant.  ·  Gourav Shah · School of DevOps & AI

M5·20