The KV cache stores every past token's keys and values in every layer. It grows with context × concurrency and can become a memory bottleneck at long context or high concurrency. These comparisons omit activation peaks, workspace and allocator overhead. At this config the KV cache (250 GiB) already exceeds the model weights (130.4 GiB) — the cache, not the weights, is what limits concurrency. Switch the attention variant to see how MHA→GQA→MQA→MLA shrink it, and try FP8/INT4 to quantize the cache itself.