Autoregressive decoding relies on the KV cache, whose memory footprint grows linearly with context length and caps batch size and throughput. Existing post-t…
机构:华为
来源:arXiv 2611.07025 | AI4Papers 论文推荐平台