Under expert parallelism, mixture-of-experts (MoE) layers pay all-to-all token dispatch, grouped GEMMs, and a combine step in every layer. In decode each ran…
机构:阿里
来源:arXiv 2611.04222 | AI4Papers 论文推荐平台