DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation

类型：arxiv
标识：2610.04596
链接：https://arxiv.org/abs/2610.04596
主分类：multimodal
形态：method
TLDR：On-policy distillation (OPD) has emerged as a widely used paradigm for post-training large language models, reducing the train--test mismatch of conventional distillation by supervising the student on its own generated trajectories. However, existing OPD objectives remain largely token-local and outcome-agnostic, optimizing teacher--student agreement at each prefix despite reasoning quality being determined at the trajectory level. Reinforcement learning with verifiable rewards (RLVR), particularly Group Relative Policy Optimization (GRPO), provides complementary outcome-level supervision but
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-10-07-agent-rag-longcontext-candidates.json

2026-10-05

攻略 nick7nlp/Awesome-LLM-On-Policy-Distillation · 上手攻略

2026-08-27

笔记 SPEAR: 训练-free 符号化过程奖励用于 RL 知识蒸馏 — 精读与批判

2026-09-01

文档 Tom 评 flyP 的产出：On-Policy Distillation 三大要点解读（2607.13399）

2026-07-20

笔记 flyP 精读与批判 · 2026-07-28 15:50

2026-07-28

笔记 AI 蒸馏不是"中段失效"——arXiv 2609.01532 用一个 entropy 路由让推理 +1.6x、事实记忆保 96.8%

2026-09-04

arXiv

2610.04596

TLDR

On-policy distillation (OPD) has emerged as a widely used paradigm for post-training large language models, reducing the train--test mismatch of conventional distillation by supervising the student on its own generated trajectories. However, existing OPD objectives remain largely token-local and outcome-agnostic, optimizing teacher--student agreement at each prefix despite reasoning quality being determined at the trajectory level. Reinforcement learning with verifiable rewards (RLVR), particularly Group Relative Policy Optimization (GRPO), provides complementary outcome-level supervision but

arXiv ↗ 中文全文

DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation

相关 · 按语义匹配