← All explorers
Loading explorer…
Read the theory: transformers
Read the theory: transformers
Pick a query token and watch softmax(QKᵀ/√d_k) decide who it listens to — slide the temperature to see the scaling that keeps gradients alive, or mask the future for autoregression.
Each row of the heatmap is one token's attention distribution: softmax(QK^T/√d_k) over all keys. Brighter cells = more attention. Pick a query token, then slide the temperature to watch softmax go from sharp (near one-hot) to flat (near uniform) — the knob behind the √d_k scaling — or toggle the causal mask.
Illustrative model for building intuition, not a live system.