← All explorers
Loading explorer…
Read the theory: transformers
Read the theory: transformers
Move the query and key positions and watch their vectors rotate. The attention score depends only on the gap between them — slide both together and it never budges.
RoPE encodes position by rotating each query and key vector by an angle proportional to its position. The dials show the query pairs (blue) and key pairs (green) spinning as you move the positions. The one number that matters — the attention score — depends only on the relative offset m−n, not on the absolute positions. That's the whole trick: slide both positions together and the score never moves; change the gap and it does.
Illustrative model for building intuition, not a live system.