Pre-Transformer Architectures

Before the Transformer swept through every modality after 2017, deep learning ran on two parallel tracks. Convolutional Neural Networks (CNNs) owned vision, from LeCun's LeNet to AlexNet's 2012 ImageNet moment to ResNet's residual revolution. Recurrent Neural Networks (RNNs) and their gated descendants (LSTM, GRU) owned sequences: speech, translation, language modeling, anything time-indexed. Both arcs share the same plot: an architectural insight unlocks new scale, scaling reveals a gradient pathology, an intervention (residual connections for CNNs, gating for RNNs) buys another order of magnitude, and eventually attention shows up and rewrites the rules in both domains. Understanding these architectures is not just history. CNNs still dominate edge inference, segmentation, and medical imaging; RNN-T variants ship in every production streaming ASR; and the Mamba/SSM line is a serious 2026 counterweight to attention.

Vision Arc: Convolutional Neural Networks

CNNs are the architecture that made deep learning practical for visual data. Before CNNs, computer vision relied on hand-engineered features (SIFT, HOG); after AlexNet's 2012 ImageNet win, the field pivoted overnight to learned hierarchical features and never looked back.

The Core Intuition

A convolutional layer slides a small filter (kernel) across the input, computing a dot product at every spatial location. Three properties make this fundamentally different from a fully connected layer:

Local receptive fields: each output neuron looks at only a small spatial neighborhood (e.g., 3×3) of the input, matching the physical reality that nearby pixels are correlated and faraway pixels usually are not.
Parameter sharing: the same kernel weights are reused at every spatial position. A filter that learns to detect a horizontal edge works equally well in any corner of the image. This collapses the parameter count from O(input_size × output_size) to O(kernel_size²) per filter.
Translation equivariance: shift the input by k pixels and the output shifts by k pixels. The network does not need to relearn what a cat looks like in different spatial locations.

These priors are baked into the architecture, which is why CNNs train well on small datasets where Transformers struggle: the architecture itself encodes "vision-like" assumptions that would otherwise have to be learned from data.

Kernel, Stride, Padding, Dilation

The four hyperparameters that define a convolution:

Kernel size (e.g., 3×3, 5×5): modern architectures overwhelmingly use stacks of 3×3 kernels because two 3×3 layers have the same receptive field as one 5×5 layer but with fewer parameters and an extra non-linearity.
Stride: how many pixels the kernel jumps per step. Stride 2 halves spatial resolution.
Padding: zeros added around the input to preserve spatial dimensions. SAME keeps output size; VALID shrinks by kernel_size - 1.
Dilation: gaps inserted between kernel elements. A 3×3 dilated conv with rate 2 has the same receptive field as a 5×5 conv with only 9 parameters. Used in DeepLab (segmentation) and WaveNet (audio).

Pooling and the Receptive Field

Pooling layers reduce spatial dimensions and add translation invariance. Max pooling picks the dominant activation; average pooling smooths; global average pooling (GAP) collapses each feature map to one number, replacing the huge FC head of LeNet/AlexNet/VGG and dramatically reducing overfitting. Modern architectures increasingly use strided convolutions instead of pooling: the network learns how to downsample.

The receptive field of a neuron is the input region it can "see":
RF_out = RF_in + (k - 1) · d · jump_in
jump_out = jump_in · stride

Stacking ten 3×3 stride-1 conv layers gives only a 21×21 receptive field. Dilation rates [1, 2, 4, 8, 16] reach a 63×63 RF with the same parameter count: this is the WaveNet/DeepLab trick.

Channel Operations: 1×1, Depthwise, Pointwise, Separable

1×1 convolutions operate purely on the channel dimension, used for bottleneck reduction (ResNet-50's 1×1 → 3×3 → 1×1 block), pointwise channel mixing, channel attention (Squeeze-and-Excitation), and cheap non-linearity.
Depthwise convolutions apply one filter per input channel: no channel mixing, only spatial filtering.
Depthwise separable convolutions = depthwise + 1×1 pointwise. For a 3×3 conv on C channels, this achieves the same expressive power as a standard conv with roughly 1/C_out + 1/9 of the FLOPs (~8-9× cheaper). The foundation of MobileNet, Xception, EfficientNet, ConvNeXt.

The Classic Architecture Ladder

LeNet-5 (1998, LeCun): first practical CNN, ~60K parameters, trained on MNIST for check digit recognition. Established the conv → pool → conv → pool → FC template that survived for two decades.
AlexNet (2012, Krizhevsky/Sutskever/Hinton): 60M parameters, 8 layers, trained on two GTX 580 GPUs. Dropped ImageNet top-5 error from 26% to 15% in one paper. Introduced ReLU (vs. tanh, which trained 6× faster), dropout, aggressive augmentation, and GPU training as a first-class concern. Kicked off the modern deep learning revolution.
VGG (2014, Simonyan & Zisserman) proved depth matters more than fancy design. Monotonous stacks of 3×3 convs and 2×2 max pools, 16 or 19 layers deep. Still used today as a feature extractor for perceptual loss.
Inception / GoogLeNet (2014): the Inception block runs 1×1, 3×3, 5×5 convs and pooling in parallel and concatenates the results. Used 1×1 convs aggressively to keep params low (6.8M vs VGG's 138M).
ResNet (2015, He et al.) is the most-cited paper in deep learning. The residual connection y = F(x) + x lets gradients flow through an identity path that bypasses every layer. With residual connections, 152, 200, even 1001-layer networks became trainable. ResNet-50 is still the default vision backbone in 2026.
DenseNet (2016): every layer connects to every subsequent layer via concatenation. Compact and feature-reusing but awkward memory access; ResNet variants stayed dominant.
EfficientNet (2019) introduced compound scaling: scale depth, width, and resolution together with a fixed ratio. EfficientNet-B7 matched ResNet-152 accuracy with 8× fewer parameters.

Why CNNs Dominated Vision Pre-2020

Three reasons ConvNets owned vision for nearly a decade: (1) the convolutional priors (locality, equivariance) act as a free regularizer, making them dramatically more data-efficient than priorless architectures, and most labeled image datasets are small. (2) Hardware-friendly memory access patterns map well to GPU tensor cores, so they train and infer fast. (3) The architecture composes naturally with task-specific heads: the same backbone serves classification, detection (Faster R-CNN, YOLO), segmentation (U-Net, Mask R-CNN), and depth estimation.

Sequence Arc: RNN, LSTM, GRU

The sequence track ran in parallel. An RNN processes a sequence one element at a time, maintaining a hidden state that summarizes everything seen so far. The defining property is weight sharing across time (the same parameters at every timestep), which makes the model agnostic to sequence length and gives it a strong inductive bias for temporal structure.

h_t = f(W_h · h_{t-1} + W_x · x_t + b)
y_t = g(W_y · h_t + b_y)

The hidden state h_t is the network's memory; the recurrence is unrolled across time and trained via Backpropagation Through Time (BPTT), where gradients flow backward through every timestep of the unrolled graph.

Vanilla RNN: Gradient Instability

Vanilla RNNs are fatally fragile across long sequences. BPTT multiplies the recurrent Jacobian ∂h_t/∂h_{t-1} at every timestep. If its spectral radius is < 1, gradients shrink exponentially (vanishing); if > 1, they blow up (exploding). Vanishing gradients prevent learning long-range dependencies: by the time the loss signal propagates back 50+ steps, it has effectively disappeared. Exploding gradients cause NaN losses; the standard fix is gradient clipping (capping the L2 norm), but that does nothing for the vanishing direction.

LSTM: Long Short-Term Memory (Hochreiter & Schmidhuber, 1997)

The LSTM solves vanishing gradients with a deliberate architectural intervention: a separate cell state c_t flowing through the sequence with only linear (additive) interactions, plus three multiplicative gates controlling what enters, leaves, and is read.

f_t = σ(W_f · [h_{t-1}, x_t] + b_f) forget gate (what to drop from cell state)
i_t = σ(W_i · [h_{t-1}, x_t] + b_i) input gate (what new info to write)
g_t = tanh(W_g · [h_{t-1}, x_t] + b_g) candidate values
c_t = f_t ⊙ c_{t-1} + i_t ⊙ g_t updated cell state (additive!)
o_t = σ(W_o · [h_{t-1}, x_t] + b_o) output gate
h_t = o_t ⊙ tanh(c_t)

The key insight is the cell-state update: c_t = f_t ⊙ c_{t-1} + i_t ⊙ g_t. When f_t ≈ 1, the gradient ∂c_t/∂c_{t-1} ≈ diag(f_t) is approximately the identity, so gradients flow backward through hundreds of timesteps without vanishing, because they pass through addition rather than repeated multiplication. This is the constant error carousel the original 1997 paper identified. In practice, biasing the forget gate toward 1 at initialization (b_f ≈ 1 or 2) dramatically speeds up learning of long-range dependencies. LSTMs dominated sequence modeling for nearly two decades: speech recognition, machine translation, handwriting, language modeling.

GRU: Gated Recurrent Unit (Cho et al., 2014)

A streamlined LSTM merging cell and hidden state, using only two gates:

z_t = σ(W_z · [h_{t-1}, x_t]) update gate (how much past to keep)
r_t = σ(W_r · [h_{t-1}, x_t]) reset gate
h̃_t = tanh(W · [r_t ⊙ h_{t-1}, x_t]) candidate
h_t = (1 - z_t) ⊙ h_{t-1} + z_t ⊙ h̃_t

GRUs have ~25% fewer parameters than LSTMs and train faster. Empirically, no consistent winner across benchmarks (Chung 2014, Greff 2016): LSTMs tend to edge out on tasks needing fine-grained long-memory gating; GRUs tend to win on small datasets where parameter efficiency matters.

Bidirectional RNNs

For offline sequence labeling (NER, POS tagging), a bidirectional RNN runs two RNNs in parallel (left-to-right and right-to-left) and concatenates their hidden states: h_t = [h_t^→ ; h_t^←]. BiLSTM-CRF was state-of-the-art for NER for years. Critical limitation: bidirectional models cannot stream, since the backward pass needs the entire sequence.

Seq2Seq with Bahdanau Attention (2014): the First Practical Attention

The pre-attention encoder-decoder (Sutskever 2014) compressed the entire source sentence into a single fixed-length hidden state vector, which the decoder unrolled into the target. This worked for short inputs but degraded catastrophically as length grew: a single vector cannot hold a 40-word sentence. Bahdanau et al. (2014) introduced attention as a fix: instead of forcing the decoder to rely on one summary vector, let it dynamically "look back" at all encoder hidden states at every decoder step, computing a weighted sum where weights are learned alignments.

α_{t,i} = softmax(score(s_t, h_i))
c_t = Σ_i α_{t,i} · h_i

Bahdanau used an additive MLP scoring function v^T tanh(W_s · s_t + W_h · h_i); Luong (2015) refined with simpler dot-product and bilinear variants. This was the first practical attention mechanism in deep learning. It produced three lasting effects: (1) BLEU jumped on long sentences; (2) attention weights gave the first interpretable view of what the model was using for each output; (3) it planted the seed for the radical next step: what if attention is enough, and we drop recurrence entirely?

The Shared Headache: Gradient Vanishing and Exploding

Both arcs hit the same wall on the way up. As CNNs got deeper and RNNs got longer, gradients refused to flow cleanly between distant layers / timesteps. The math is the same in both cases: chain-rule products of Jacobians whose eigenvalues either contract or expand exponentially with depth.

Why Deep CNNs Suffered (and What Motivated ResNet)

Before ResNet, stacking more conv layers made networks worse: a 56-layer plain network had higher training error than a 20-layer one. This was the degradation problem: not overfitting (training error was worse, not just test error) and not pure vanishing gradients (BatchNorm had largely solved that). The real issue was that the identity function was apparently hard for stacked nonlinear layers to learn, so adding layers prevented the deep network from even matching the shallower one. Residual connections (y = F(x) + x) solved this with two mechanisms: (1) reparametrization, where the network learns the residual F(x) = H(x) - x instead of the full mapping, and driving F(x) → 0 is much easier than driving F(x) → x; (2) gradient highway, where the identity path means gradients have a direct route to every earlier layer that doesn't pass through any non-linearity. Writing the chain rule: dL/dx_l = dL/dx_L · (1 + d/dx_l Σ F_i), the "1" term guarantees gradient magnitude never collapses. This unlocked 100+ layer training.

Why Long-Sequence RNNs Suffered (and What Motivated LSTM Gating)

The vanilla RNN Jacobian ∂h_t/∂h_{t-1} = W_h^T · diag(tanh'(...)) is multiplied across T timesteps during BPTT. Spectral radius < 1 → gradients vanish at the rate of the largest eigenvalue raised to the power T; spectral radius > 1 → gradients explode. By T = 50, signal is dead. The LSTM's intervention is exactly the recurrent analog of ResNet's identity path: the additive cell-state update c_t = f_t ⊙ c_{t-1} + i_t ⊙ g_t has Jacobian diag(f_t) along the memory channel. When the forget gate is near 1, this is the identity, the constant error carousel. The deep parallel: both architectures fix gradient pathology by giving signal a near-identity path that bypasses the nonlinear compositions.

The Standard Toolkit

The fixes that emerged are now universal across deep learning:

Residual / skip connections (ResNet, Highway Networks, U-Net's encoder→decoder skips, every Transformer block): identity paths for gradients.
Gating (LSTM, GRU, GLU, SwiGLU in modern Transformers): learned multiplicative controls that let the network decide how much to mix old and new signal.
Gradient clipping fixes exploding gradients (cap global L2 norm at 1 to 5); essential for RNNs, still used as a safety net even for Transformers.
Normalization: BatchNorm (CNNs, batch ≥ 16), LayerNorm (Transformers, modern CNNs, any batch size), GroupNorm (small-batch detection/segmentation), RMSNorm (modern LLMs). Stabilizes activations and gradients across layers / timesteps.
Better initialization: He / Glorot initialization keeps activation variances stable through stacked nonlinearities; biasing LSTM forget gate to ~1 starts the model with healthy long-range gradient flow.

The Transition to Attention

By 2017, both arcs had architectural fixes for the gradient problem and were producing solid results. Then attention came for both.

Why Transformers Replaced RNNs for Language

Three structural problems made RNNs untenable at the scale modern language modeling demanded:

No parallelism over the time dimension. Every h_t depends on h_{t-1}, so you cannot evaluate h_5 without first computing h_1, ..., h_4 sequentially. Modern GPUs/TPUs have tens of teraFLOPs of matmul throughput, but only when work is parallelizable. A 100M-parameter LSTM took weeks to train on what a 100M-parameter Transformer trained in days; at 100B parameters, the gap becomes "impossible vs. tractable." The Transformer's self-attention is a single QK^T matmul over the whole sequence, fully parallel across positions.
Long-range dependencies remain hard even with gating. LSTMs improved on vanilla RNNs by orders of magnitude but still struggle beyond a few hundred timesteps. Self-attention gives every position direct access to every other position in O(1) sequential steps: the path length between any two tokens collapses from O(n) to O(1), making gradient flow trivial.
Scaling laws favored attention. Once GPT-2/GPT-3 demonstrated that Transformer loss scales smoothly with parameters, data, and compute, the empirical case for RNNs collapsed; there was no comparable scaling story.

Bahdanau attention had already shown attention worked as a layer on top of RNNs. "Attention Is All You Need" (Vaswani et al., 2017) made the bolder leap: drop recurrence entirely.

Why ViT Replaced CNNs for Vision at Scale

Vision was the last holdout; CNN inductive biases seemed irreplaceable for pixel data. The Vision Transformer (ViT, Dosovitskiy 2020) showed otherwise: split a 224×224 image into 16×16 patches (196 patches), flatten each into a vector, linearly project to d_model, add learned positional embeddings, prepend a [CLS] token, and feed the sequence into a standard Transformer encoder. No convolutions, no pooling.

On ImageNet-1k from scratch, ViT underperformed ResNets: without convolutional priors, it needed more data to learn that neighboring pixels are correlated. But pre-trained on JFT-300M (300M images), ViT matched or beat the best CNNs at comparable accuracy with fewer FLOPs. The deeper lesson: CNNs hard-code priors that are helpful with limited data but limiting with abundant data. Once you have enough examples to learn relationships between any two image regions directly via attention, the convolutional locality assumption becomes a ceiling rather than a floor. ViT also unified vision and NLP under one architecture, which is why every modern multimodal model (CLIP, DALL-E 3, Flamingo, Gemini, GPT-4V) uses ViT-style image tokenization, letting the same Transformer stack process text and image tokens jointly.

The 2026 Reality: ConvNeXt and State-Space Models as Counterweights

The honest 2026 picture is more nuanced than "attention won everything."

ConvNeXt (Liu et al., 2022, "A ConvNet for the 2020s") pushed back on the CNN side: by porting ViT's training recipe and design choices (AdamW, stochastic depth, layer scale, larger kernels, GELU, LayerNorm, fewer activations per block) into a pure-CNN architecture, they matched or beat Swin Transformer on ImageNet across all model sizes. Much of ViT's headline performance came from training tricks, not from attention itself. CNNs still dominate: low-data regimes (<100K images), edge / mobile inference (MobileNet, EfficientNet-Lite), semantic segmentation (U-Net is the universal default for medical imaging and Stable Diffusion's backbone), classical detection (YOLO families), and medical imaging.

Mamba and Selective State-Space Models (Gu & Dao, 2024) are the recurrence revival on the sequence side. An SSM is a continuous-time linear recurrence h'(t) = A · h(t) + B · x(t); y(t) = C · h(t), discretized into h_t = Ā · h_{t-1} + B̄ · x_t. Structured S4 (Gu 2022) used HiPPO initialization and a convolutional reformulation to enable parallel training, achieving Transformer-quality results on Long Range Arena with linear sequence-length cost. Mamba's contribution is making the A, B, C matrices input-dependent: a selection mechanism that gives the SSM something analogous to attention's content-based focus while preserving O(1) recurrent inference and constant-size state (no KV cache). Hybrid architectures like Jamba (AI21, 2024) interleave Mamba and attention blocks for best-of-both-worlds long context. RNN-T variants still ship in every production streaming ASR (Google, Apple, Amazon) for the same reason.

The deeper takeaway: recurrence was not dead; it was being done wrong in the LSTM era. SSMs are recurrence redesigned for GPUs, with parallel scans for training and stable long-range dynamics from the continuous-time formulation. Convolutions were not dead either: ConvNeXt showed they just needed the same training-time love attention models got. The pre-Transformer architectures still matter, in both their classical forms (where their priors and efficiency win) and their modern descendants (where their core ideas have been redesigned).