Mixture-of-Experts (MoE) models are bottlenecked by loading routed-expert weights from memory during interactive decoding at small batch sizes. Because most …
机构:KAIST
来源:arXiv 2610.37386 | AI4Papers 论文推荐平台