Agentic serving can consume orders of magnitude more tokens than chatbot workloads, stressing both KV-cache capacity and decode-time bandwidth. Most KV-evict…
机构:帝国理工
来源:arXiv 2609.14872 | AI4Papers 论文推荐平台