RAG and Vector Search Systems

Understand why retrieval is used with language models and how embeddings make semantic search possible.

• Why RAG: knowledge freshness, grounding, and limits of fine-tuning
• The pipeline stages: ingest, chunk, embed, index, retrieve, generate
• What an embedding is and how similarity is computed
• Hosted embedding APIs versus open models in Sentence Transformers
• Embedding dimensions, cost, and multilingual considerations
• Failure modes: retrieval misses versus generation errors

Hands-on lab: Embed a small document set with a hosted and an open model, run the same ten queries against both, and compare the top results by hand.

Turn messy real-world documents into chunks that retrieve well.

• Parsing PDFs, HTML, and office documents
• Fixed-size, recursive, and structure-aware chunking
• Chunk overlap and its tradeoffs
• Attaching metadata such as source, section, and date
• Handling tables and images in documents
• Deduplication and incremental re-ingestion

Hands-on lab: Ingest a mixed PDF and HTML corpus with two chunking strategies and compare retrieval of ten reference questions under each.

Work hands-on with the main options for storing and querying vectors and learn how to choose among them.

• pgvector: tables, vector columns, and SQL queries
• Qdrant collections, payloads, and filtering
• Pinecone indexes and namespaces
• Approximate nearest neighbor search: HNSW and IVF concepts
• Metadata filtering and its effect on recall
• Choosing a store: operational fit, scale, cost, and existing stack

Hands-on lab: Load the same embedded corpus into pgvector, Qdrant, and Pinecone, run identical filtered queries, and record setup effort, latency, and result differences.

Improve relevance by combining lexical and semantic signals and reordering candidates.

• Keyword search with BM25 and where it beats embeddings
• Combining result lists with reciprocal rank fusion
• Cross-encoder rerankers versus bi-encoder retrieval
• Query rewriting and expansion
• Choosing candidate set size and the final top k
• Latency impact of each added stage

Hands-on lab: Add BM25 and a reranker to your baseline retriever and measure the change in recall and latency on a fixed query set.

Build the answer-generation step so responses stay grounded in retrieved content.

• Assembling context within token limits
• Prompts that instruct the model to use only supplied sources
• Returning citations and quoting passages
• Handling no-answer and conflicting-source cases
• Conversational RAG and follow-up question rewriting
• Frameworks versus hand-written pipelines: LlamaIndex and LangChain

Hands-on lab: Build an answering service that returns citations, refuses when evidence is missing, and handles a follow-up question using conversation history.

Measure retrieval and generation separately so you know what to fix.

• Building a question set with reference passages
• Retrieval metrics: recall at k, precision, and mean reciprocal rank
• Generation metrics: faithfulness, answer relevance, and context relevance
• Using Ragas and LLM-based judges, and their limits
• Error analysis: sorting failures into retrieval, context, and generation causes
• Tracking quality across changes

Hands-on lab: Create a 30-question evaluation set, score your pipeline with retrieval metrics and Ragas, and write a failure analysis with the top three fixes.

Prepare a RAG system for real users, changing data, and real constraints.

• Keeping the index fresh with scheduled and event-driven updates
• Document-level access control and multi-tenant filtering
• Caching embeddings and answers
• Monitoring retrieval quality and user feedback
• Prompt injection through retrieved content
• Index migrations when changing the embedding model

Hands-on lab: Add per-user access filters and an update job to your system, then test that a restricted document never appears for an unauthorized user.