Deep Learning and NLP Foundations

Build a network from tensors and autograd, then make training stable and learn to read loss curves.

• Tensors, devices, GPU use, and autograd
• Layers, activations, loss functions, and backpropagation
• Dataset, DataLoader, and a complete training loop
• Optimizers such as SGD with momentum and AdamW, and learning rate schedules
• Batch normalization, dropout, weight decay, and overfitting diagnosis
• Experiment tracking with Weights & Biases

Hands-on lab: Implement a multilayer perceptron with a hand-written training loop, then deliberately break training with a poor learning rate and missing normalization and fix it step by step while tracking runs.

Apply convolutional architectures to images and reuse pretrained models effectively.

• Convolution, pooling, and receptive fields
• Common architectures such as ResNet
• Data augmentation with torchvision transforms
• Transfer learning: feature extraction versus fine-tuning
• Handling small and imbalanced image datasets
• Inspecting errors with confusion matrices and example review

Hands-on lab: Fine-tune a pretrained ResNet on a small image dataset, apply augmentation, and review the misclassified examples for patterns.

Understand how text becomes numbers and how recurrent models handle sequences.

• Tokenization and vocabularies
• Bag-of-words and TF-IDF baselines
• Word embeddings such as word2vec and GloVe
• Recurrent networks, LSTMs, and GRUs
• Padding, masking, and batching variable-length text
• Why recurrent models struggle with long dependencies

Hands-on lab: Build a TF-IDF logistic regression baseline and an LSTM classifier on the same text dataset and compare accuracy, training time, and errors.

Learn the architecture behind modern language models by building its core pieces.

• Self-attention with queries, keys, and values
• Multi-head attention and positional encoding
• Encoder, decoder, and encoder-decoder designs
• Pretraining objectives for BERT-style and GPT-style models
• Subword tokenization such as BPE and WordPiece
• Computational cost and context length

Hands-on lab: Implement a single self-attention block in PyTorch, visualize its attention weights on example sentences, and explain what each head appears to capture.

Adapt pretrained transformers to your own tasks using the Hugging Face ecosystem.

• Loading models, tokenizers, and datasets from the Hub
• Sequence classification and token classification
• The Trainer API and custom training loops
• Parameter-efficient fine-tuning with LoRA through the PEFT library
• Handling long inputs and class imbalance
• Reading model cards and checking licenses

Hands-on lab: Fine-tune a pretrained transformer for text classification, then repeat it with LoRA and compare quality, memory use, and training time.

Use sentence embeddings to find and group text by meaning.

• Sentence embeddings with sentence-transformers
• Cosine similarity and normalization
• Approximate nearest neighbor search with FAISS
• Clustering and visualizing embeddings
• Chunking strategies for long documents
• Evaluating retrieval with recall at k and mean reciprocal rank

Hands-on lab: Build a semantic search index over a document set, then measure its retrieval quality against a hand-labeled query set.

Evaluate models beyond a single headline score, then package a trained model so it can be served reliably.

• Metrics for classification, tagging, and generation such as F1, seqeval, and ROUGE
• Slicing results by subgroup and input type
• Systematic error analysis and train-test contamination checks
• Exporting to ONNX and serving predictions with FastAPI
• Latency, batching, and quantization basics

Hands-on lab: Run a sliced evaluation of your fine-tuned model and categorize fifty errors, then wrap the classifier in a FastAPI service in Docker and measure latency with and without ONNX export.