Smaller Models, Better Rejects: Preference Distillation Scaling

类型：arxiv
标识：2609.38987
链接：https://arxiv.org/abs/2609.38987
主分类：llm-infra
形态：method
TLDR：Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come from a model at least as large as the student, making generation costly at scale. We find neither assumption holds: across students from 7B to 72B, smaller frozen models generate rejects with less inference compute yet train stronger students than self-generated rejects, before and after sequence-level knowledge distillation, on code generation and mathematical reasoning. To e
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-10-02-agent-rag-longcontext-candidates.json

2026-08-11

攻略 nick7nlp/Awesome-LLM-On-Policy-Distillation · 上手攻略

2026-08-27

文档 Tom 评 flyP 的产出：On-Policy Distillation 三大要点解读（2607.13399）

2026-07-20

文档 Stephen 自测 · R67 · 2026-09-06

2026-09-06

笔记 AI 蒸馏不是"中段失效"——arXiv 2609.01532 用一个 entropy 路由让推理 +1.6x、事实记忆保 96.8%

2026-09-04

笔记 SPEAR: 训练-free 符号化过程奖励用于 RL 知识蒸馏 — 精读与批判

2026-09-01

arXiv

2609.38987

TLDR

Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come from a model at least as large as the student, making generation costly at scale. We find neither assumption holds: across students from 7B to 72B, smaller frozen models generate rejects with less inference compute yet train stronger students than self-generated rejects, before and after sequence-level knowledge distillation, on code generation and mathematical reasoning. To e

Smaller Models, Better Rejects: Preference Distillation Scaling

相关 · 按语义匹配