🧠 Deep Learning Basics

Deep learning basics are the mechanics that make neural networks train: backpropagation, activation functions, weight initialization, normalization (BatchNorm and LayerNorm), optimizers (SGD vs Adam), and regularization. Interviews test whether you understand why training diverges, plateaus, or overfits, not just the formulas.

Forward Pass & Backprop

A neural network is a composition of differentiable functions: y = f_L(f_{L-1}(...f_1(x))). The forward pass evaluates this composition layer by layer, caching intermediate activations. The backward pass computes gradients of a scalar loss with respect to every parameter by applying the chain rule in reverse order, multiplying the loss's upstream gradient by each layer's local Jacobian as we walk back through the cached activations.

Modern frameworks (PyTorch, JAX, TF) implement this via reverse-mode automatic differentiation. Each tensor operation is a node in a dynamic computational graph; the framework records the graph during the forward pass, then traverses it in reverse to compute vector-Jacobian products (VJPs). We never materialize full Jacobians: for a layer mapping ℝⁿ → ℝᵐ, the Jacobian would be n×m, but we only need J^T · v where v is the upstream gradient, which is a single matrix-vector product. This is why autograd scales: gradient cost is O(forward cost), not O(forward × params).

Activations (sigmoid, tanh, ReLU family, GELU, SwiGLU)

Non-linearities give networks their expressive power; a stack of linear layers collapses to one linear layer. The choice of activation has changed dramatically over 15 years:

• Sigmoid σ(x) = 1/(1+e⁻ˣ): saturates at 0 and 1, gradients ≤ 0.25. Stacked sigmoids cause vanishing gradients through the chain rule (0.25⁵ ≈ 10⁻³). Mostly dead outside the output layer of binary classifiers.
• Tanh: zero-centered version of sigmoid, gradients ≤ 1. Slightly better than sigmoid but still saturates.
• ReLU max(0, x): gradient is 1 or 0, no saturation in the positive regime. The default for CNNs and many MLPs since AlexNet (2012). Failure mode: dead neurons. Once a unit's pre-activation goes negative for all training inputs, its gradient is 0 forever and it never recovers.
• Leaky ReLU / PReLU: small negative slope (e.g., 0.01x) to avoid dead neurons.
• ELU / SELU: smooth negative tails with self-normalizing properties.
• GELU x·Φ(x): smooth, used in BERT, GPT-2/3.
• SwiGLU (Swish-Gated Linear Unit): SwiGLU(x) = (xW · swish(xV)), a gated formulation where one linear projection modulates another via swish (x·σ(x)). Used in LLaMA, PaLM, and most modern LLMs because the gating mechanism gives the FFN more expressiveness per parameter; empirically yields better loss at fixed compute. SwiGLU FFNs use ~8/3 × d_model hidden size (vs 4× for ReLU) to keep parameter count matched.
• Mish: x·tanh(softplus(x)), smooth, non-monotonic, occasionally beats GELU on vision.

Vanishing / Exploding Gradients

The chain rule multiplies Jacobians: ∂L/∂W₁ = (∂L/∂a_L)(∂a_L/∂a_{L-1})...(∂a_2/∂a_1)(∂a_1/∂W_1). If each factor has spectral norm < 1, the product decays exponentially with depth (vanishing); if > 1, it explodes. RNNs hit this catastrophically across timesteps: the same recurrent matrix is multiplied at every step, so its spectral radius governs whether information from t=0 still has a gradient signal at t=1000.

Mitigations at the architecture/training level: ReLU-family activations (avoid sigmoid saturation), careful initialization (next section), normalization layers, residual connections (LSTMs, ResNets, Transformers all rely on them so gradients can flow through identity paths), and gradient clipping. The deep dive into how LSTMs, GRUs, and gated recurrent units specifically tackle the problem (and why attention sidesteps it entirely) lives in the Pre-Transformer Architectures topic.

Initialization (Xavier, He)

Starting weights badly is its own way to make gradients vanish or explode. Two scaling rules dominate:

• Xavier / Glorot init: Var(W) = 2 / (fan_in + fan_out). Derived assuming linear or tanh activations; keeps the variance of activations and gradients roughly constant across layers.
• He / Kaiming init: Var(W) = 2 / fan_in. Correct for ReLU, which zeros out half the inputs in expectation, so we compensate by doubling the variance.
• Orthogonal init: weight matrices initialized as random orthogonal matrices. Preserves norms exactly under linear maps, useful for RNNs and very deep networks.

The fan-in/fan-out reasoning: at layer l, output variance ≈ fan_in · Var(W) · Var(input). Setting Var(W) = c/fan_in keeps Var(output) ≈ c · Var(input), preventing per-layer blowup or collapse. Modern transformers add a residual-branch scaling trick (divide residual-path weights by 1/√(2N) where N is the number of layers) so the residual stream variance doesn't grow with depth.

Normalization (BatchNorm, LayerNorm, RMSNorm, GroupNorm)

• BatchNorm (Ioffe & Szegedy, 2015): normalize each feature across the batch dimension using batch mean/variance during training; use running averages during inference. Powerful for CNNs but has problems: train/eval discrepancy (different statistics), brittle at small batch sizes (variance estimates noisy), breaks when batch composition matters (contrastive learning, RL), and awkward across distributed training. Cannot be used cleanly in autoregressive sequence models because the batch dimension mixes information across sequences.
• LayerNorm (Ba et al., 2016): normalize across the feature dimension per example. No batch dependence, identical train/eval, works at any batch size. The default in transformers.
• RMSNorm (Zhang & Sennrich, 2019): LayerNorm without the mean-subtraction step, just dividing by RMS. Same quality, ~7-15% faster, fewer parameters. Used in LLaMA, PaLM, most modern LLMs.
• GroupNorm (Wu & He, 2018): split channels into groups and normalize per group. Used in vision tasks where batch sizes are small (detection, segmentation).

LLMs use LayerNorm/RMSNorm because they're sequence-length and batch-size invariant, critical when sequence lengths vary per batch and inference batch sizes can be 1.

Optimization (SGD, momentum, Adam, AdamW, Lion; LR schedules; gradient clipping; mixed precision)

SGD updates θ ← θ − η · ∇L. Simple but slow and noisy. SGD + momentum maintains a velocity vector v ← βv + ∇L; θ ← θ − η · v. Averages out noise and accelerates along consistent directions. Nesterov momentum evaluates the gradient at the look-ahead point θ − η · βv, giving slightly better convergence.

Adaptive optimizers scale per-parameter learning rates by gradient statistics:

• Adagrad: divide by sqrt of accumulated squared gradients. Helps rare features but learning rate decays monotonically to zero.
• RMSprop: exponential moving average of squared gradients (fixes Adagrad's monotonic decay).
• Adam (Kingma & Ba, 2014): maintains first moment m (EMA of gradients) and second moment v (EMA of squared gradients), applies bias correction m̂ = m/(1−β₁ᵗ), v̂ = v/(1−β₂ᵗ) to undo initialization bias, then updates θ ← θ − η · m̂ / (√v̂ + ε). Default for most deep learning.

AdamW (Loshchilov & Hutter, 2017): decoupled weight decay. In Adam + L2, the L2 term ends up in the gradient and gets scaled by the adaptive 1/√v̂ factor, so parameters with large gradients get less weight decay, which is the opposite of what regularization should do. AdamW applies decay directly to the parameters (θ ← θ − η · (m̂/√v̂ + λθ)), decoupling it from the gradient. This matters enormously for transformers: training large LLMs with Adam+L2 leads to poor generalization and unstable late-stage training; AdamW is the de-facto standard.

Lion (Chen et al., 2023): uses only the sign of the momentum-smoothed gradient. ~2× less memory than Adam (no second moment), comparable quality on large-scale training. Sophia (2023): incorporates diagonal Hessian estimates for second-order-ish updates, ~2× faster pretraining on LLMs.

Learning rate schedules: Cosine with linear warmup is the current standard for LLM pretraining: linearly ramp up over ~1-2K steps (avoids instability early when momentum/variance estimates are noisy), then decay following half a cosine to a small final LR. Other schedules: step decay (drop by 10× at fixed epochs, classical CNN choice), polynomial decay (BERT), 1cycle (super-convergence for shorter runs), WSD (warmup-stable-decay) for cooldown phases on additional data.

Gradient clipping: clip-by-norm (rescale g to have norm ≤ τ, preserving direction) is preferred over clip-by-value (clamp each element). Critical for transformers: a single spike (e.g., from a degenerate batch) can blow up the optimizer state and corrupt training. Typical norm threshold: 1.0.

Mixed precision: store weights in FP32 (master copy) but compute in FP16 or BF16. BF16 has the same exponent range as FP32 (no overflow issues), making it the default for LLM training; FP16 has higher precision but tiny dynamic range, requiring loss scaling (multiply loss by ~2¹⁶ before backward, divide gradients back) to prevent underflow. FP8 (Hopper/Blackwell GPUs) pushes further with per-tensor scaling. Throughput typically 2-4× over FP32 with minimal quality loss.

Loss Functions (CE, MSE, MAE, Huber, hinge, contrastive, focal, KL)

• Cross-entropy for classification: -Σ y · log p. Connects to maximum likelihood estimation: minimizing CE = maximizing log-likelihood of training labels under the model. For binary: BCE = -[y log p + (1−y) log(1−p)].
• MSE (L2) for regression: smooth, penalizes large errors heavily, but sensitive to outliers. MAE (L1): robust to outliers but non-smooth at zero. Huber: quadratic near zero, linear far from it, robust and smooth, used in RL (DQN) and bounding-box regression.
• Hinge loss max(0, 1 − y·f(x)) for max-margin classification (SVMs).
• Contrastive losses: InfoNCE pulls together positive pairs and pushes apart negatives within a batch (foundation of CLIP, SimCLR, contrastive sentence embeddings). Triplet loss max(0, d(a,p) − d(a,n) + α) operates on anchor, positive, and negative triples (FaceNet).
• Focal loss (Lin et al., 2017): -(1−p)^γ log p, which downweights well-classified examples by (1−p)^γ. Used in dense object detection (RetinaNet) and any heavily imbalanced classification setup.
• KL divergence D_KL(P||Q) = Σ P log(P/Q): measures how P differs from Q. Asymmetric: D_KL(P||Q) ≠ D_KL(Q||P). Forward KL (P||Q) is mode-covering (Q must support P's modes); reverse KL (Q||P) is mode-seeking. Used in variational inference (ELBO), knowledge distillation (student matches teacher distribution), RLHF (KL penalty against reference model).

Regularization in DL (dropout, weight decay, label smoothing)

Classical regularization theory (L1/L2 penalties, bias-variance tradeoff, cross-validation) lives in the ML Fundamentals topic. Here we cover the three regularizers that matter most in deep nets:

• Dropout (Srivastava et al., 2014): randomly zero out a fraction p of activations during training, then scale the rest by 1/(1−p) (inverted dropout). Forces the network to not rely on any single unit, approximating an ensemble of exponentially many sub-networks. Standard p=0.1-0.5 for MLPs/CNNs. Modern LLMs often use dropout=0 during pretraining (the data scale is its own regularizer) but apply small dropout (0.05-0.1) during fine-tuning. Variants: DropPath / stochastic depth drops entire residual branches (used in ViT, EfficientNet); attention dropout drops attention weights post-softmax.
• Weight decay (decoupled, via AdamW): adds λθ to the parameter update, shrinking weights toward zero. Typical λ = 0.1 for LLM pretraining. As discussed under optimization, decoupled weight decay is critical: never use Adam + L2 for large models.
• Label smoothing (Szegedy et al., 2016): replace one-hot targets y with a smoothed distribution (1−ε)·y + ε/K (where K is the number of classes, typical ε=0.1). Prevents the model from becoming overconfident, improves calibration, and slightly improves accuracy on classification tasks. Used in the original Transformer paper, image classification SOTA models, and most production classifiers. Tradeoff: hurts top-1 accuracy by a hair while improving calibration substantially.

Other deep-net regularizers worth knowing: early stopping (monitor validation loss, halt before overfitting), data augmentation (the strongest regularizer for vision: random crops, flips, mixup, cutmix), and noise injection (Gaussian noise on inputs or activations).

Sequence Modeling: a brief storyline

Modeling sequences (language, audio, time series) sits at the heart of why deep learning matters. The architectural arc from 2000 to 2017 is a tight chain of "this hurts → fix that → new pain → fix that" moves. We sketch it here as motivation; the architecture deep-dive lives in Pre-Transformer Architectures.

n-grams (pre-2010). The classical answer was to count: estimate P(word_t | word_{t−n+1}, ..., word_{t−1}) from corpus frequencies, with Kneser-Ney smoothing to handle unseen n-grams. Simple, fast, and surprisingly competitive: Google's 2007 trillion-word 5-gram model held translation records for years. But two walls: (1) sparsity, where vocabulary^n parameter space explodes past n=5, so context windows stayed tiny; (2) no generalization across surface forms, so "the cat sat" and "the kitten sat" share no parameters, meaning semantically similar contexts can't pool statistics.

RNNs (Elman 1990; revived by Mikolov ~2010). Map each token to a learned embedding, then propagate a hidden state h_t = tanh(W_h · h_{t−1} + W_x · x_t) through the sequence. Now context is unbounded in principle, and similar words have similar embeddings, so generalization arrives. But the chain rule through hundreds of identical W_h multiplications produces the vanishing/exploding gradient problem in its most painful form: by the time backprop reaches t=0 from t=100, the signal has either decayed to noise or blown up to NaN.

LSTM (Hochreiter & Schmidhuber, 1997; rediscovered post-2014) and GRU. Add a cell state with multiplicative gates (input, forget, output) that let the network choose whether to remember, overwrite, or reveal information at each step. The forget gate is the critical innovation: when its value is near 1, the cell state propagates almost unchanged, giving gradients a near-identity highway over hundreds of timesteps. LSTMs powered Google Translate (2016), speech recognition, and the first wave of useful sequence models. But two limits remained: (1) sequential bottleneck, where each step waits for the previous, blocking GPU parallelism along the time dimension; (2) fixed-size hidden state, since all of "the document so far" must be squeezed into one vector, which becomes a brutal information bottleneck for long contexts.

Attention (Bahdanau et al., 2014; Vaswani et al., 2017). Bahdanau let the decoder softly look back at every encoder hidden state instead of relying on the final one, solving the bottleneck while still wrapped around an RNN. Vaswani took the bold next step: drop recurrence entirely. With self-attention, every position queries every other position in a single matmul, so the path length between distant tokens collapses from O(n) to O(1), and the whole sequence parallelizes across GPUs. Training throughput jumped ~10× overnight, and the scaling laws kicked in. The Transformer (covered in its own topic) won everything from there.

The lesson: each transition was driven by removing the bottleneck created by the previous fix. n-grams couldn't generalize → embeddings + RNNs. RNN gradients died → LSTM gates. LSTM still couldn't parallelize → attention. Modern revivals (Mamba, RWKV, state-space models) try to reclaim recurrent O(n) compute while keeping attention's training parallelism, so the storyline isn't over.