Lab

Mon, 01 Jan 0001 00:00:00 +0000

Time: ~25 min | Day 2 PM (shared with M07)

Learning Objectives

Build a RAG-based knowledge assistant. Ingest documents, query with grounding, and compare RAG vs. non-RAG responses.

Concept Review: RAG in Practice

The RAG Pipeline

1. INGEST: Documents → chunks → embeddings → vector DB
2. QUERY: User question → embed → search vector DB → retrieve top-K chunks
3. AUGMENT: Build prompt with retrieved chunks as context
4. GENERATE: Send augmented prompt to LLM → grounded response

Key Components

Component	What It Does	Options
Embedding Model	Converts text to vectors	Chromadb built-in (onnx, free), OpenAI embeddings
Vector Database	Stores and searches embeddings	ChromaDB (lightweight, pip install), Pinecone (cloud)
Chunking Strategy	How you split documents	Fixed-size (200-500 tokens), sentence-based, semantic
Top-K Retrieval	How many chunks to retrieve	Start with 3-5; more = broader context but higher token cost

Why RAG Instead of Fine-Tuning?

RAG	Fine-Tuning
Dynamic — docs update instantly	Static — retrain when data changes
No model training needed	Requires training data and compute
Works with any LLM	Tied to a specific model
Provides citations (retrieved chunks)	No built-in provenance
Cheaper and faster to implement	Better for consistent style/tone shifts

Common Failure Modes

Retrieval misses relevant docs → Chunks too large or too small; try different chunk size
Irrelevant chunks confuse the LLM → Lower top-K; add relevance threshold
Context window overflow → Too many chunks; reduce top-K or chunk size
Stale data → Vector DB not re-indexed; schedule regular ingestion

Prerequisites

LLM API access (for embeddings and generation)
ChromaDB: pip install chromadb
Sample documents provided in workshop/m06/docs/

Using OpenCode or VS Code Chat: Both work for this lab. OpenCode is good for executing the Python scripts. VS Code Chat is good for explaining concepts as you go.

M06: Retrieval-Augmented Generation (RAG) on Generative AI Boot Camp for Developers

Lab

Learning Objectives

Concept Review: RAG in Practice

The RAG Pipeline

Key Components

Why RAG Instead of Fine-Tuning?

Common Failure Modes

Prerequisites