The Key-Value (KV) cache is the main memory bottleneck of long-context LLM inference. Token eviction methods such as SnapKV, H2O, AdaKV and L2Norm shrink it,…
机构:三星
来源:arXiv 2610.25431 | AI4Papers 论文推荐平台