Prefix caching avoids prefill only when a repeated request returns to a server that still holds the prefix KV. Cache-blind balancing disperses that reuse; fi…
机构:Meta
来源:arXiv 2608.19677 | AI4Papers 论文推荐平台