The KV cache stores every past token's keys and values in every layer. It grows with context × concurrency and is usually the real memory bottleneck at serving time. At this config the KV cache (250 GB) already exceeds the model weights (130.4 GB) — the cache, not the weights, is what limits concurrency. Switch the attention variant to see how MHA→GQA→MQA→MLA shrink it, and try FP8/INT4 to quantize the cache itself.