📐 ML Fundamentals

ML fundamentals are the statistical core every interview probes: probability and Bayes, the bias-variance tradeoff, overfitting, cross-validation, and regularization. Mastering them lets you reason about why a model generalizes, where it fails, and how to fix it, the judgment interviewers actually test.

Probability & Bayes

Probability is the mathematical language of uncertainty, and it underpins nearly every ML concept, from loss functions to Bayesian inference to generative models. Before you can reason about a model's generalization, calibration, or failure modes, you need a fluent grip on conditional probability, priors, and how evidence updates belief.

Key Definitions

Sample space (Ω): The set of all possible outcomes. For a coin flip, Ω = {H, T}.

Event: A subset of the sample space. P(event) is always between 0 and 1.

Conditional probability: P(A|B) = P(A ∩ B) / P(B), the probability of A given that B has occurred.

Independence: A and B are independent if P(A ∩ B) = P(A) · P(B), equivalently P(A|B) = P(A).

Bayes' Theorem

Bayes' theorem relates conditional probabilities:

P(A|B) = P(B|A) · P(A) / P(B)

In ML terminology:

• P(A) = prior: our belief about A before seeing data
• P(B|A) = likelihood: probability of observing data B if A is true
• P(A|B) = posterior: updated belief about A after seeing data B
• P(B) = evidence (marginal likelihood), the total probability of observing B

Confusion Matrix & Classification Rates

Before the base-rate trap, we need the vocabulary for how a test can be right or wrong. Every prediction on a binary problem falls into one of four cells, comparing what the model predicted against the actual truth:

	Actually Positive	Actually Negative
Predicted Positive	True Positive (TP)	False Positive (FP)
Predicted Negative	False Negative (FN)	True Negative (TN)

Read the four cells as plain outcomes:

• TP = correctly caught a real positive (sick person flagged sick)
• TN = correctly cleared a real negative (healthy person cleared)
• FP = false alarm: flagged a negative as positive (healthy person told they're sick)
• FN = miss: let a real positive slip through (sick person cleared)

From these four counts come the rates the base-rate problem uses — each is just "of a given true class, what fraction did we get right?":

• Sensitivity = True Positive Rate (TPR) = Recall = TP / (TP + FN) — of all actual positives, the fraction we caught.
• Specificity = True Negative Rate (TNR) = TN / (TN + FP) — of all actual negatives, the fraction we correctly cleared.
• False Positive Rate (FPR) = FP / (FP + TN) = 1 − Specificity — of all actual negatives, the fraction we falsely flagged.
• Precision = TP / (TP + FP) — of everything we predicted positive, the fraction that was really positive.

A one-line way to keep sensitivity and specificity straight: sensitivity measures how well a test finds the disease (few misses), specificity measures how well it rules the disease out (few false alarms).

The Base Rate Problem

The most common interview trap with Bayes' theorem. Example: A disease test has 99% sensitivity (true positive rate) and 99% specificity (true negative rate). If the disease prevalence is 1 in 10,000, what is the probability someone with a positive test actually has the disease?

P(disease|positive) = P(positive|disease) · P(disease) / P(positive)
= 0.99 × 0.0001 / (0.99 × 0.0001 + 0.01 × 0.9999)
≈ 0.0098 ≈ 1%

Despite 99% accuracy, only ~1% of positive results are true positives. The low base rate (prior) overwhelms the test accuracy. The same trap reappears any time you have severe class imbalance (fraud detection, rare-disease screening, ad click prediction), and is why aggregate accuracy is a useless metric in those domains.

Common Distributions

• Bernoulli/Binomial: Binary outcomes (coin flips, click/no-click)
• Gaussian (Normal): Bell curve, appears everywhere due to Central Limit Theorem
• Poisson: Count of events in a fixed interval (page visits per hour)
• Exponential: Time between events (time until next click)
• Categorical/Multinomial: Discrete outcomes with >2 classes (word prediction)
• Beta: Prior over a probability in [0, 1], conjugate to Bernoulli/Binomial
• Dirichlet: Prior over a categorical distribution, conjugate to Multinomial

Key Probability Rules

• Sum rule: P(A) = Σ P(A, B_i) over all B_i (marginalization)
• Product rule: P(A, B) = P(A|B) · P(B) = P(B|A) · P(A)
• Law of total probability: P(A) = Σ P(A|B_i) · P(B_i)
• Chain rule: P(A,B,C) = P(A|B,C) · P(B|C) · P(C)

Bayesian vs Frequentist

In the frequentist view, probability is the long-run frequency of events. Parameters are fixed but unknown. Confidence intervals = "if we repeated this experiment many times, 95% of intervals would contain the true value."

In the Bayesian view, probability represents degrees of belief. Parameters have distributions. We start with a prior, update with data (likelihood), and get a posterior. Credible intervals = "there is a 95% probability the parameter is in this interval."

Most ML training is frequentist (maximize likelihood), but Bayesian concepts appear in regularization (priors), uncertainty estimation, and Bayesian neural networks. The bridge becomes important in the next section: regularization itself is a maximum a posteriori (MAP) estimator under a particular prior.

Bias-Variance Tradeoff

The bias-variance tradeoff is the fundamental concept that explains why models fail to generalize. Every model's prediction error on unseen data decomposes into three parts:

Error = Bias² + Variance + Irreducible Noise

Bias² is how far the average prediction sits from the truth (underfitting). Variance is how much predictions swing across different training sets (overfitting). Irreducible noise (Bayes error) is inherent randomness no model can remove. The walkthrough below builds each idea visually — scroll through it to see how the two failure modes trade off.

Ensembling and the Tradeoff

Two canonical ensemble strategies attack different parts of the decomposition. Bagging (random forests) reduces variance by averaging many decorrelated high-variance learners. Each tree sees a bootstrap sample and a random feature subset, so individual errors cancel out. Boosting (XGBoost, LightGBM) reduces bias by sequentially fitting shallow, high-bias trees to the residuals of the previous round, additively building up expressiveness. Stacking trains a meta-model on the out-of-fold predictions of base learners, attacking both bias and variance.

The Modern Regime: Double Descent

Classical bias-variance theory predicts that past a certain model complexity, test error always increases. Modern deep learning reveals a "double descent" curve: as you go past the interpolation threshold (where the model perfectly fits training data), test error can decrease again. Heavily overparameterized models (like large neural nets) can generalize well despite having more parameters than data points, especially with proper regularization and optimization. Double descent extends, rather than overturns, the classical theory: the U-shape still exists in the underparameterized regime, and the second descent depends on implicit regularization from SGD plus explicit weight decay.

Overfitting & Underfitting

The walkthrough above defined underfitting (high bias) and overfitting (high variance) and their fixes. Two things remain worth knowing: how to diagnose which one you have, and the surprising cases.

Diagnosing via the Train-Validation Gap

The single most useful diagnostic is the gap between training and validation error. Underfitting shows up as both errors high with a small gap; overfitting as low training error but a large gap to validation. Two cases people miss:

• Both errors low: The happy path. Ship it, but verify on out-of-distribution slices before celebrating.
• Val error << train error: Suspicious. Likely causes: label leakage in features, a validation set drawn from an easier distribution, or batch-norm statistics misuse.

Learning Curves

Plot training and validation error as a function of training-set size. The shapes tell you which problem you have:

• High bias: Both curves plateau at a high error, close together. Adding data does not lower either. You are capacity-limited, not data-limited.
• High variance: Training error stays low, validation error decreases as data grows but a large gap remains. Adding data will continue to narrow the gap. You are data-limited.
• Healthy fit: Validation error converges toward training error as data grows, both at a low value.

This is why learning curves are the first thing to plot when a model surprises you: they pin down whether the bottleneck is the model class or the dataset.

Concrete Fix Patterns

For high bias, in cheapest-first order: (1) Remove or reduce regularization (drop λ, lower dropout, disable early stopping). (2) Engineer features: interaction terms, polynomial features, embeddings for categoricals. (3) Increase model capacity (deeper, wider, more trees). (4) Train longer, since many "high bias" diagnoses are really under-trained models with adaptive optimizers. (5) Switch model family entirely.

For high variance, in cheapest-first order: (1) Add L1/L2 regularization or increase its strength. (2) Add dropout (for neural nets) or early stopping. (3) Simplify the model (shallower trees, narrower nets, fewer features). (4) Augment data: flips/crops/mixup for vision, paraphrasing/back-translation for text. (5) Get more labelled data, the most expensive but most reliable fix.

Beyond Overfitting: Distribution Shift

A model that overfits the training set will fail on a held-out validation set drawn from the same distribution. A model that fails in production while passing validation typically has a different problem: distribution shift. Three flavours: covariate shift (input distribution P(X) changes), label shift (P(Y) changes, e.g., fraud rate rises), and concept drift (P(Y|X) changes, meaning the relationship itself moves). The cure is monitoring (KS-test or PSI on feature distributions, prediction-distribution drift, slice-level performance) plus regular retraining, not more regularization.

Cross-Validation

A held-out validation set gives you one estimate of generalization error, which can be high-variance for small datasets and tempting to overfit through repeated hyperparameter tuning. Cross-validation averages multiple held-out estimates to give a more reliable signal.

k-Fold

Split the data into k equal partitions. For each fold i ∈ {1, …, k}, train on the other k-1 folds and evaluate on fold i. Average the k validation scores. Standard choices are k = 5 or k = 10, the sweet spot between estimate stability (larger k = less variance in the score) and compute cost (k× the training time of a single fit). Each example appears in exactly one validation fold and k-1 training folds, so all data contributes to both training and evaluation.

Stratified k-Fold

Standard k-fold splits randomly, which can produce class-imbalanced folds when the dataset itself is imbalanced, fatal for small minority classes (a fold might contain zero positive fraud examples). Stratified k-fold preserves the class proportions in every fold by sampling within each class. Use it by default for any classification problem with non-trivial class imbalance; the cost over random k-fold is essentially zero.

Time-Series CV (Walk-Forward / Rolling Window)

For temporal data, random splits leak future information into training, fatal in finance, demand forecasting, ads, and any setting where the production distribution is "what comes next." Use forward-chaining splits: train on [1..t], validate on [t+1..t+h], roll forward, repeat. Never include a validation index earlier than a training index. Two common variants: expanding window (training set grows with each split) and sliding window (fixed-size training set rolls forward). Sliding window better matches non-stationary regimes where recent data is most predictive; expanding window uses all history and is preferable when the underlying dynamics are stable.

Leave-One-Out (LOOCV)

The extreme case where k = n: train on n-1 examples, evaluate on the held-out one, repeat n times. Gives an almost-unbiased estimate of generalization error but has high variance (all training sets are nearly identical, so the n error estimates are correlated) and costs n full training runs. Only practical for very small datasets or for models with closed-form leave-one-out estimates (e.g., linear regression's PRESS statistic). For most problems, 5- or 10-fold is the better choice.

Nested CV

When you use cross-validation to both select hyperparameters and estimate generalization error, you contaminate the estimate: the chosen hyperparameters are biased toward this particular CV split. Nested CV fixes this with an outer loop for performance estimation and an inner loop for model selection. For each outer-fold training set, run k-fold CV inside it to pick hyperparameters, then evaluate the chosen model on the outer-fold validation set. Average across outer folds for the final, unbiased estimate. It is computationally expensive (k_outer × k_inner trainings) but is the principled approach when reporting research results or comparing model families.

Group k-Fold

When examples are not independent (multiple records per patient, multiple frames per video, multiple search queries per user), random splitting puts correlated examples in both train and validation, leaking information and inflating CV scores. Group k-fold ensures all examples sharing a group ID land in the same fold. The pattern: pick the group definition that matches the production scenario (a new patient, a new video, a new user) and stratify on it.

Common Pitfalls

• Tuning hyperparameters on the test set. The test set must be used once, at the end. If you tune on it, you have a second validation set, not a test set.
• Leakage via preprocessing. Standardize/normalize inside each fold using only the fold's training data, not across the whole dataset before splitting. Same for imputation, target encoding, and feature selection.
• Forgetting groups. If your data has natural groups (users, patients, time), random k-fold gives overly optimistic scores.
• Random splits on time series. Mentioned above; the most common production failure mode.
• Reading CV scores as a guarantee. They estimate generalization to the same distribution as training. Out-of-distribution behaviour requires separate monitoring and offline shift tests.

Regularization

Regularization is any technique that improves generalization at the cost of training fit. Without it, a model with sufficient capacity will memorize training data, driving training loss toward zero while validation loss explodes. The bias-variance tradeoff is the lens through which all regularization should be understood: we deliberately introduce bias (constraints on the hypothesis space) to reduce variance (sensitivity to the particular training sample).

In classical ML on tabular data, regularization is often the single most important hyperparameter. In modern deep learning (especially LLM pretraining), explicit regularization has become less central, replaced by scale, data diversity, and architectural choices (normalization, residual streams). But it remains critical for fine-tuning, low-data regimes, and any deployed model that must generalize beyond its training distribution.

L1 vs L2 Penalties

The two canonical penalties add a term to the loss:

L_total = L_data + λ · ||w||_p

• L2 (Ridge) uses ||w||_2² = Σ w_i². The gradient of the penalty is 2λw, proportional to the weight itself, so it shrinks weights smoothly toward (but never exactly to) zero. Geometrically the constraint region ||w||_2 ≤ C is a circle (sphere in higher dimensions), whose smooth boundary is generically tangent to loss contours at points with all coordinates non-zero. In Bayesian terms, L2 is MAP estimation with a zero-mean Gaussian prior on the weights, where λ corresponds to the inverse prior variance.

• L1 (Lasso) uses ||w||_1 = Σ |w_i|. The subgradient is λ · sign(w), a constant push toward zero regardless of magnitude, which produces exact sparsity. Geometrically the constraint region is a diamond (cross-polytope). Its corners lie on the coordinate axes, and loss contours generically touch the diamond at a corner, driving some weights to exactly zero. This is why L1 doubles as a feature-selection mechanism. In Bayesian terms, L1 is MAP with a Laplace prior.

• ElasticNet mixes both: λ_1 ||w||_1 + λ_2 ||w||_2². Useful when you want sparsity (L1) but L1 alone is unstable under correlated features (it picks one and drops the others arbitrarily); L2 stabilizes the selection.

Weight Decay vs L2 Regularization

These are often conflated, but they differ subtly and importantly. Weight decay is the update rule w ← (1 - η·λ) · w - η · ∇L_data. L2 regularization adds λ ||w||_2² to the loss, giving gradient update w ← w - η · (∇L_data + 2λw). For vanilla SGD, the two are mathematically equivalent (with a factor-of-2 reparameterization of λ).

For adaptive optimizers like Adam, they diverge. Adam scales gradients by an estimate of the second moment v; if you fold the L2 penalty into the gradient, the penalty also gets scaled by 1/√v, so weights with historically small gradients get less shrinkage than weights with large gradients, the opposite of what you usually want. AdamW (Loshchilov & Hutter, 2019) fixes this by applying weight decay directly to the weights after the Adam update, decoupled from the gradient. This is why AdamW is the default optimizer for modern transformer training.

Dropout

Dropout (Srivastava et al., 2014) randomly zeros activations during training with probability p (typically 0.1-0.5). It can be viewed as training an exponential ensemble of subnetworks that share weights, and as injecting multiplicative Bernoulli noise that prevents co-adaptation of features.

Inverted dropout is the standard implementation: scale activations by 1/(1-p) at training time so that no scaling is needed at inference. This keeps the inference path identical to a model without dropout, simplifying deployment.

Dropout fell out of favor in convolutional vision models once BatchNorm became standard: BN already injects noise (via mini-batch statistics) and the two interact awkwardly. In Transformers, dropout is still present (commonly 0.1 on attention weights and residual connections during pretraining) but often disabled entirely for the largest LLMs, where data diversity does the regularization.

Early Stopping

The simplest and arguably most powerful regularizer: monitor validation loss and stop training when it stops improving (with some patience). This implicitly limits the effective hypothesis class (the model has fewer optimizer steps to fit noise) and is closely connected to L2 regularization via the early-stopping equals L2 result in linear models. Always save the best checkpoint (lowest validation loss), not the last one.

Label Smoothing

Replace one-hot targets [0, 0, 1, 0] with smoothed targets [ε/K, ε/K, 1-ε+ε/K, ε/K] (typically ε = 0.1). This prevents the model from becoming infinitely confident (without smoothing, cross-entropy pushes the correct logit toward +∞), which improves calibration and often generalization. Used in the original Transformer paper and in image classification (Inception-v3 onward). Caveat: label smoothing interacts poorly with knowledge distillation (the teacher's soft probabilities already encode similar information, and smoothing can suppress the useful inter-class signal).

Data Augmentation as Regularization

Any transformation that preserves the label is a form of regularization: random crops, flips, color jitter, mixup (convex combinations of inputs and labels), CutOut (zeroing random patches), and learned policies like RandAugment / AutoAugment. Data augmentation is often more effective than explicit weight penalties because it adds task-relevant invariances rather than generic shrinkage.

Batch / Layer Normalization

Normalization layers regularize as a side effect. BatchNorm computes mean/variance from the mini-batch, so each training example sees slightly different statistics, equivalent to multiplicative + additive noise on activations. LayerNorm/RMSNorm don't have this stochasticity (they normalize per-example) but stabilize training, which lets you use higher learning rates and stronger augmentation.

Modern LLM Regularization

For frontier LLMs the recipe has shifted: most explicit regularization is dropped during pretraining (or set to a tiny value), and generalization is achieved through (1) scale, (2) data diversity and deduplication, (3) careful learning-rate scheduling, (4) AdamW weight decay (typically 0.1), and (5) architectural regularizers like QK-norm or z-loss that stabilize attention. During fine-tuning, where data is scarce, classical regularization (LoRA's implicit low-rank constraint, dropout in attention, early stopping, smaller learning rates) is much more relevant. The RLHF KL penalty against a reference policy is itself a regularizer preventing distribution drift.