← All explorers
Loading explorer…
Read the theory: transformers
Read the theory: transformers
Full attention scores every token against every other — n² work. Grow the sequence and watch the cost explode, then switch to windowed or sparse patterns to tame it.
The grid is the attention matrix: one cell per (query, key) pair. Full attention fills all n² of them; sliding-window and sparse patterns compute only a fraction. Sweep the sequence length and watch full attention's cost explode quadratically while the sparse patterns stay nearly linear — that gap is why long-context models exist.
Illustrative model for building intuition, not a live system.