Scaled weight decay for Muon (Muon-SW) has been reported to train language models more efficiently than constant decoupled weight decay, but such comparisons…
机构:Microsoft
来源:arXiv 2611.02529 | AI4Papers 论文推荐平台