The KV cache has become a major bottleneck in deploying LLMs, as its memory footprint grows linearly with sequence length and batch size, imposing substantia…
机构:腾讯
来源:arXiv 2609.19880 | AI4Papers 论文推荐平台