引入 student reference KL 损失并 mask 特殊终止 token 的 advantage,以缓解生成过长和频繁截断的问题;在 HLE 和 HiPhO 等科学 benchmark 上取得改进,表明 OPD 传递的推理能力可泛化至数学训练领域之外。This work introduces a student reference KL loss and mask the advantages of special termination tokens to mitigate the problem of excessive generation length and frequent truncation, and improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain.
论文
148 张论文卡片 · 工程化 · 方法
提出 Mimir v1,一个基于 Hierarchical Reasoning Model(HRM)架构的 10 亿参数语言模型,从头训练,在英语上具有高度竞争力,并仅使用合规的后训练数据在丹麦语上创下新的 SOTA。Mimir v1 is introduced, a 1-billion-parameter language model based on the Hierarchical Reasoning Model (HRM) architecture that is trained from scratch and delivers highly competitive performance for English and sets a new state of the art for Danish using only permissible post-training data.
提出 LITTLECURRICULUM,一个为美国小学课程量身定制的 88B token 精选预训练语料,并由此训练得到 LITTLELEARNER;该模型具备开放式评估所需的语言能力,同时其知识和能力边界清晰可映射到可解释的课程指南。This work introduces LITTLECURRICULUM, a curated 88B-token pretraining corpus tailored to U.S. elementary school material, which yields LITTLELEARNER, a model with sufficient language competence for open-ended evaluation, yet with clear knowledge and capability boundaries mapped to interpretable curriculum guidelines.
提出一个即插即用的 2D Motion Interface,使预训练于 3D 的 MoLM 能够在不修改或微调原始模型的情况下接受 2D 运动输入,并在 2D 运动任务上优于从头训练 MoLM。A plug-and-play 2D Motion Interface is introduced that enables 3D-pretrained MoLMs to accept 2D motion inputs without modifying or fine-tuning the original models and outperforms training MoLMs from scratch on 2D motions.