MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation
- 类型：arxiv
- 标识：2609.30837
- 链接：https://arxiv.org/abs/2609.30837
- 主分类：engineering
- 形态：position
- TLDR：Multi-teacher on-policy distillation (MOPD) integrates specialized capabilities into a single student, but existing practice typically hard-routes each prompt to a domain-matched teacher for the entire rollout. This dependence on prompt-level domain labels restricts using unlabeled training mixtures and leaves complementary signals from other teachers unused. We introduce MOPD-Router, a framework that routes supervision over the full teacher pool at each token, without domain labels or training a separate routing model. Its plug-in interface supports different metrics for selecting and weighti
- 待LLM分类：否
- 标题中文：[标题中文] MOPD-Router：重新思考多教师在策略蒸馏中的教师路由
- TLDR中文：[TLDR中文] 多教师在策略蒸馏（MOPD）可将多种专有能力集成到单一学生模型中，但现有方法通常在整条 rollout 期间将每个 prompt 硬路由到领域匹配的教师。这种对 prompt 级领域标签的依赖限制了无标注训练混合数据的利用，并使其他教师的互补信号未被使用。我们提出 MOPD-Router，一个在每个 token 上对全体教师池进行路由监督的框架，无需领域标签，也无需训练单独的路由模型。其插件式接口支持不同的指标来选择和加权
- 来源文件：
- /inbox/tom/_candidates/2026-09-28-agent-rag-longcontext-candidates.json