论文

本文提出 UniMoMo，一种后训练压缩框架，将其形式化为约束图粗化问题，并引入分层自适应保护机制，根据路由暴露度限制对高流量 expert 的合并。UniMoMo, a post-training compression framework formulated as a constrained graph coarsening problem, is introduced, and a layer-adaptive protection mechanism that restricts the merging of high-traffic experts based on their routing exposure is introduced.

本工作提出在海量文本语料上使用新的自监督目标 PEGASUS 对大型 Transformer 编码器-解码器模型进行预训练，并证明其在所有 12 个下游数据集上按 ROUGE 分数衡量均取得 SOTA 性能This work proposes pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective, PEGASUS, and demonstrates it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores.

从算法建模、应用和理论分析角度对 MTL 的综述，给出了 MTL 的定义，并将不同 MTL 算法分为五类：特征学习方法、低秩方法、任务聚类方法、任务关系学习方法和分解方法A survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses, which gives a definition of MTL and classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach,task relation learning approach and decomposition approach.

较为全面地列举了基于群体智能、生物启发、物理启发和化学启发（按灵感来源分类）的所有算法，这些算法已成为解决实际问题的流行工具A relatively comprehensive list of all the algorithms based on swarm intelligence, bio-inspired, physics-based and chemistry-based, depending on the sources of inspiration, that have become popular tools for solving real-world problems.

提出了能够平衡"延迟告警的上下文敏感成本"与"打断成本"的模型与推理流程，并通过对用户活动与通知内容的分析，描述了在不确定性下推理此类成本所面临的挑战Models and inference procedures that balance the context-sensitive costs of deferring alerts with the cost of interruption are presented and the challenge of reasoning about such costs under uncertainty via an analysis of user activity and the content of notifications is described.

本文提供了参数空间探索能够改进 LLM 强化学习的证据，并提出称为扰动参数策略优化（3PO）的方法族，使用不同的采样策略和不同的 rollout 分组进行 reward 估计。Evidence that parameter-space exploration can improve reinforcement learning for LLMs is presented, and a family of methods called Perturbed Parameter Policy Optimization (3PO) is introduced which use different sampling strategies and different rollout grouping for reward estimation.

本工作训练了一个预测的计算最优模型 Chinchilla，使用与 Gopher 相同的计算预算，但参数量为 70B、数据量为 4 倍，达到 SOTA 平均准确率，比 Gopher 提升超过 7%。This work trains a predicted compute-optimal model, Chinchilla, that uses the same compute budget as Gopher but with 70B parameters and 4$\times$ more more data, and reaches a state-of-the-art average accuracy, greater than a 7% improvement over Gopher.

一种具有固定大小记忆的循环 Transformer 架构，可泛化 sliding-window attention，同时在训练期间保持并行性，并在验证 loss 和下游预训练 benchmark 上优于 sliding-window 和 latent recurrent Transformer 基线。A recurrent Transformer architecture with fixed-size memory that generalizes sliding-window attention while remaining parallelizable during training and improves validation loss and downstream pretraining benchmarks over sliding-window and latent recurrent transformer baselines.

本文提出一种权重裁剪的替代方案：对 critic 相对于其输入的梯度范数施加惩罚。其性能优于标准 WGAN，能以几乎无需调参的方式稳定训练多种 GAN 架构。This work proposes an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input, which performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning.

首个同时使用 LLM 推理和 tag-aware 翻译来明确处理并评估英罗 MT 中性别偏见的方法。This is the first method to explicitly address and evaluate gender bias in English-Romanian MT using both LLM inference and tag-aware translation.

引入 student reference KL 损失并 mask 特殊终止 token 的 advantage，以缓解生成过长和频繁截断的问题；在 HLE 和 HiPhO 等科学 benchmark 上取得改进，表明 OPD 传递的推理能力可泛化至数学训练领域之外。This work introduces a student reference KL loss and mask the advantages of special termination tokens to mitigate the problem of excessive generation length and frequent truncation, and improves on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain.

提出 Mimir v1，一个基于 Hierarchical Reasoning Model（HRM）架构的 10 亿参数语言模型，从头训练，在英语上具有高度竞争力，并仅使用合规的后训练数据在丹麦语上创下新的 SOTA。Mimir v1 is introduced, a 1-billion-parameter language model based on the Hierarchical Reasoning Model (HRM) architecture that is trained from scratch and delivers highly competitive performance for English and sets a new state of the art for Danish using only permissible post-training data.

提出 LITTLECURRICULUM，一个为美国小学课程量身定制的 88B token 精选预训练语料，并由此训练得到 LITTLELEARNER；该模型具备开放式评估所需的语言能力，同时其知识和能力边界清晰可映射到可解释的课程指南。This work introduces LITTLECURRICULUM, a curated 88B-token pretraining corpus tailored to U.S. elementary school material, which yields LITTLELEARNER, a model with sufficient language competence for open-ended evaluation, yet with clear knowledge and capability boundaries mapped to interpretable curriculum guidelines.

发现面向推理的训练并未优先放大具有最高 Lift 的行为，这促使研究者采用过程级目标，以奖励经过校准且有依据的推理，而不仅仅是表面形式。It is found that reasoning-oriented training does not preferentially amplify the highest-Lift behaviors, motivating process-level objectives that reward calibrated and grounded reasoning rather than surface form alone.

提出一个即插即用的 2D Motion Interface，使预训练于 3D 的 MoLM 能够在不修改或微调原始模型的情况下接受 2D 运动输入，并在 2D 运动任务上优于从头训练 MoLM。A plug-and-play 2D Motion Interface is introduced that enables 3D-pretrained MoLMs to accept 2D motion inputs without modifying or fine-tuning the original models and outperforms training MoLMs from scratch on 2D motions.