Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents
Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents
本文表明强化学习（RL）后训练已具备实现有效 step-level 评分所需的要素，从而完全无需额外的奖励模型训练，并在通用随机 Markov 决策过程下推导出一种隐式 advantage，称为 progress advantage。This work shows that reinforcement learning (RL) post-training already provides the ingredients for effective step-level scoring, eliminating the need for dedicated reward model training altogether, and derives an implicit advantage under a general stochastic Markov decision process, which is term progress advantage.
Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
通过表征锚定与语言-动作对齐的可泛化 VLA 微调
本文提出 Anchor-Align，通过两个目标增强 BC：Vision-Language Anchoring 从冻结 VLM 副本中蒸馏逐层表示以防止该漂移；Language-Action Alignment 将每个动作目标转换为离散的运动方向标签，并在同一机器人观测上联合训练语言与动作预测。Anchor-Align is proposed, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, and Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation.
DataPrep-Bench: Benchmarking LLMs as Training Data Preparators
DataPrep-Bench: 将 LLM 作为训练数据准备器的基准测试
提出 DataPrep-Bench，首个统一基准，在共享的下游任务 grounding 协议下，对 LLM 驱动的数据准备在六个领域、多种 base model 上的两类能力进行联合评估。DataPrep-Bench is introduced, the first unified benchmark that jointly evaluates both capabilities under a shared downstream-grounded protocol over six domains and multiple base models of LLM-driven data preparation.