Building LLM Applications

Make your first calls to the major provider APIs and understand the request, response, and token model underneath them.

• Chat message roles, system prompts, and conversation state managed by the client
• OpenAI, Anthropic, and Gemini SDKs compared side by side
• Tokens, context windows, and max output settings
• Sampling parameters such as temperature and top-p
• Rate limits, timeouts, retries, and error handling
• Keeping API keys out of source control

Hands-on lab: Write a small client that sends the same prompt to three providers, logs token usage and latency for each, and handles a forced rate-limit error gracefully.

Treat prompts as versioned code: structured, testable, and parameterized.

• Separating instructions, context, and user input
• Few-shot examples and when they help
• Prompt templates and variable injection
• Prompt injection basics and why input must be treated as untrusted
• Versioning prompts alongside application code
• Multimodal inputs such as images and PDFs

Hands-on lab: Refactor a hard-coded prompt into a versioned template with examples, then run it against ten varied inputs and record where it breaks.

Make model output machine-readable and let models invoke your functions in a controlled way.

• JSON mode versus schema-constrained structured outputs
• Defining schemas with Pydantic or Zod-style validation
• Repair and retry loops for invalid output
• Function and tool definitions across providers
• The tool-call loop: model request, your execution, result returned
• Parallel tool calls and tool error handling

Hands-on lab: Build an extraction service that returns schema-validated JSON from messy text, then add two tools the model can call and test a case where a tool fails.

Deliver responsive experiences and structure an LLM feature inside a real service.

• Server-sent events and token streaming
• Streaming with tool calls and structured output
• Backend API design with FastAPI
• Conversation memory and context trimming strategies
• Background jobs and batch APIs for non-interactive work
• Timeouts, cancellation, and fallback providers

Hands-on lab: Expose your extraction and chat features through a FastAPI endpoint that streams tokens to a simple client and cancels cleanly when the client disconnects.

Understand where money and time go in an LLM request and how to reduce both without hurting quality.

• Input versus output token pricing and how to calculate request cost
• Prompt caching offered by providers
• Application-level caching of repeated requests
• Choosing smaller or larger models per task
• Reducing latency with shorter prompts, streaming, and parallelism
• Usage tracking and budget alerts

Hands-on lab: Instrument your service to log per-request tokens and cost, then apply two optimizations and report the measured before-and-after difference.

Replace guesswork with repeatable tests that tell you whether a change made things better or worse.

• Building a golden dataset from real examples
• Deterministic checks versus model-graded evaluation
• LLM-as-judge: strengths, biases, and calibration against human labels
• Regression testing prompts and model upgrades
• Running evaluations in CI
• Online feedback and sampling production traffic

Hands-on lab: Create a 30-example evaluation set for your service, write automated scorers, and run it against two prompt versions to choose between them.

Ship the feature with guardrails, monitoring, and a deployment you can maintain.

• Input and output validation and content moderation endpoints
• Defending against prompt injection and data leakage
• PII handling and redaction before sending data to a provider
• Containerizing the service with Docker
• Logging, tracing, and alerting for model calls
• Handling provider outages and model deprecations

Hands-on lab: Containerize your service, add a redaction step and an output check, and run a short red-team session where peers try to make it misbehave.