MODULE 5  ·  DAY 2

Docs Assistant — Naive RAG

Grounding an LLM in your own documents

Question about YOUR docs Retrieved context Grounded answer find generate Retrieval-Augmented Generation — no fine-tuning, just wiring

Gourav Shah  ·  School of DevOps & AI  ·  Hands-on

M5·01

What you'll learn

The anatomy of a real GenAI app — and how to wire it in containers.

12 34 5 Name the four parts of every GenAI application Understand a vector DB as search by meaning, not by title Walk the naive-RAG pipeline: ingest → chunk → embed → retrieve → generate Wire it in Compose — ChromaDB + app talking to native Ollama See where naive RAG breaks — setting up Module 6

By the end you have a containerised Docs Assistant that answers grounded questions about Acme's runbooks.

M5·02

A model alone can't answer about YOUR docs

It answers confidently from training data that predates your runbooks — and gets it wrong.

Raw LLM — ungrounded "How do I restart the payments service?" plausible-sounding but WRONG command RAG — grounded same question + the actual runbook kubectl rollout restart deploy/payments — correct add docs

RAG gives the model a cheat sheet at query time — it generates from context, not from stale weights.

M5·03

Anatomy of a GenAI application

Four parts, wired together. You already built the first two in M2 and M3.

Application Streamlit UI · orchestrate LLM endpoint qwen2.5:1.5b · Ollama Embedding model nomic-embed-text · 768-dim Vector database ChromaDB · search by meaning already have it (M2/M3) compose grows by 2 services

LLM endpoint + embedding model + vector DB + application — M5 adds the vector DB and the app layer.

M5·04

A vector DB shelves by meaning, not title

A filing cabinet needs the exact name. A librarian understands what you mean.

Filing cabinet (by title) "Payments Runbook" "SRE ops, tier 1" "Deploy notes" wrong words → nothing found Librarian (by meaning) ? question → coordinates → nearest shelf

Embeddings are coordinates that encode meaning — similar passages land near each other, so similarity search finds them.

M5·05

Embeddings: text becomes coordinates

An embedding model turns each passage into 768 numbers that place it in meaning-space.

"restart payments" "bounce the service" "the moon is round" Embedding model 768-dim space near far

The same model embeds both your documents and your questions — that shared space is what makes retrieval work.

M5·06

The naive-RAG pipeline

Ingest once. Then answer every question with the same embedding step.

INGEST · run once Load doc Chunk Embed Store → ChromaDB QUERY · every question Embed query Retrieve top-3 Augment prompt Generate same embedding model

Load → chunk → embed → store, then embed → retrieve → augment → generate. The embedding step is shared.

M5·07

Where the pieces run

Models stay native for Metal; the vector store and app are containers.

Native Ollama :11434 qwen2.5:1.5b — generate nomic-embed-text Docker containers ChromaDB :8000 Streamlit app host.docker .internal

The M2 pattern continues: model servers native, everything else containerised over the host bridge.

M5·08

ChromaDB — the lightest vector store

Zero-config and under 2 GB — the right default for a 16 GB laptop. Scale up when you outgrow it.

ChromaDB — default zero-config · one service Python-native · <10 lines persistent · ≤ 2 GB total pinned to 0.5.20 scale up Qdrant millions of vectors · filtering pgvector already on Postgres? add a column

The API you learn — add_documents, similarity_search — is nearly identical across every alternative.

M5·09

Wired via environment variables

Hand-author the compose service by service. Endpoints are just env vars — portable across runtimes.

compose.yaml model → Ollama OLLAMA_URL=host.docker.internal embeddings EMBED_MODEL=nomic-embed-text ChromaDB CHROMA_HOST=chromadb:8000 app (Streamlit) reads all of the above one block at a time

Swap the endpoint values and the same stack runs anywhere — the wiring is configuration, not code.

M5·10

Learning Mode — watch the pipeline run

The app surfaces each step live, with timings, so the invisible parts of RAG become visible.

12 34 Query embedding Similarity search Retrieved context LLM generation 768-dim vector + ms taken searched N, got top-3 the exact runbook lines gen time + which model

Trace which sentence in which document produced the answer — the single best tool for building RAG intuition.

M5·11

Where naive RAG breaks down

It works when the question matches the wording. It fails in predictable ways.

Query mismatch — phrasing differs, misses the chunk Wrong chunk boundary — a 500-char split cuts a procedure Single-pass retrieval — one miss, no retry No query rewriting — jargon and typos hurt scores Stale index — docs changed, index didn't Module 6 Agentic RAG fixes every row

An agent can rewrite queries, retry retrieval, and judge sufficiency — understanding these failures motivates M6.

M5·12

TO THE LAB

Retrieve, then generate — grounded.

"restart payments?" the exact runbook chunk kubectl rollout restart deploy retrieve generate

Hand-author the compose, ingest Acme's runbooks, and watch Learning Mode reveal the answer.

Next: Lab — build the Docs Assistant.  ·  Gourav Shah · School of DevOps & AI

M5·13