Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation

类型：arxiv
标识：2609.02998
链接：https://arxiv.org/abs/2609.02998
主分类：multimodal
形态：method
TLDR：On-policy distillation (OPD) accelerates post-training by providing dense token-level supervision from a frozen teacher on the student's own rollouts. Vanilla OPD applies this supervision uniformly across prompts, without checking whether the teacher is reliable for each prompt. Because reverse KL is mode-seeking, a confidently wrong teacher can induce a strong yet misleading update. Distributional proxies, such as entropy or teacher-student likelihood agreement, measure uncertainty or agreement but do not directly verify outcome correctness. We introduce Teacher-Gated On-Policy Distillation (
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-09-08-agent-rag-longcontext-candidates.json

2026-08-11

攻略 nick7nlp/Awesome-LLM-On-Policy-Distillation · 上手攻略

2026-08-27

笔记 SPEAR: 训练-free 符号化过程奖励用于 RL 知识蒸馏 — 精读与批判

2026-09-01

笔记 AI 蒸馏不是"中段失效"——arXiv 2609.01532 用一个 entropy 路由让推理 +1.6x、事实记忆保 96.8%

2026-09-04

文档 Tom 评 flyP 的产出：On-Policy Distillation 三大要点解读（2607.13399）

2026-07-20

文档 Stephen 自测 · R67 · 2026-09-06

2026-09-06

arXiv

2609.02998

TLDR

On-policy distillation (OPD) accelerates post-training by providing dense token-level supervision from a frozen teacher on the student's own rollouts. Vanilla OPD applies this supervision uniformly across prompts, without checking whether the teacher is reliable for each prompt. Because reverse KL is mode-seeking, a confidently wrong teacher can induce a strong yet misleading update. Distributional proxies, such as entropy or teacher-student likelihood agreement, measure uncertainty or agreement but do not directly verify outcome correctness. We introduce Teacher-Gated On-Policy Distillation (

Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation

相关 · 按语义匹配