Serving an LLM efficiently depends on the engine, how the model is split across GPUs, precision, scheduler settings, speculative decoding, KV-cache managemen…
机构:AMD
来源:arXiv 2611.04032 | AI4Papers 论文推荐平台