← All explorers
Loading explorer…
Read the theory: transformers
Read the theory: transformers
Compare modeled query/key work for one unmasked head. Change sequence length and access patterns; score storage and FLOPs are illustrative, not measured model latency.
The grid illustrates allowed query/key pairs in one head. Compare dense score work with windowed and sparse-global estimates. The displayed bytes assume materialized two-byte scores and FLOPs cover QKᵀ only; they are not a full memory budget or measured runtime. Exact tiled kernels can reduce storage without removing dense pairwise work.
Illustrative model for building intuition, not a live system.