On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so sys…
机构:剑桥
来源:arXiv 2611.11450 | AI4Papers 论文推荐平台