Calibrating Teacher--Student Discrepancy for On-Policy Distillation
- 类型：arxiv
- 标识：2609.21619
- 链接：https://arxiv.org/abs/2609.21619
- 主分类：engineering
- 形态：method
- TLDR：On-policy distillation (OPD) improves reasoning models by learning the token-level discrepancy between a stronger teacher and an on-policy student. However, this discrepancy does not purely reflect the capability gap between the teacher and the student: it also contains deviations arising from the teacher itself, which are consequently mixed into the observed teacher--student discrepancy and indiscriminately learned by standard OPD during training. This issue is further exacerbated by privileged OPD, where privileged information induces larger teacher-side likelihood shifts, thereby encouragin
- 待LLM分类：否
- 来源文件：
- /inbox/tom/_candidates/2026-09-21-agent-rag-longcontext-candidates.json