RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning

类型：arxiv
标识：2609.20784
链接：https://arxiv.org/abs/2609.20784
主分类：agent
形态：method
TLDR：Multi-turn agents trained with reinforcement learning (RL) receive a single scalar reward per trajectory, which motivates self on-policy distillation (OPD) to supply dense token-level supervision from a self-teacher with privileged task skills, letting a skill-free student internalize them. This recipe, however, is undermined by two findings in agentic tasks: privileged information alone does not always make a teacher reliable, and the benefit of teacher supervision is stage-dependent. We therefore propose RetireOPD (Self-Retiring On-Policy Distillation), which first optimizes a decoupled, ski
副分类：multimodal
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-09-18-agent-rag-longcontext-candidates.json

2026-09-01

笔记 ContextRL: Context-Aware RL for Agentic and Multimodal LLMs

2026-06-17

笔记深度精读 · ContextRL: Context-Aware RL for Agentic and Multimodal LLMs（arXiv:2606.17053）

2026-07-04

笔记批判性精读 · OraRL "Annotations as Rollouts" `arXiv:2608.20492` · 视频 MLLM 强化学习样本效率新范式

2026-08-27

攻略 Direct-OPD：小模型 RL 策略转移，大模型无需重跑 · 干货攻略

2026-07-20

笔记 flyP 精读与批判 · 2026-07-28 15:50

2026-07-28

arXiv

2609.20784

TLDR

Multi-turn agents trained with reinforcement learning (RL) receive a single scalar reward per trajectory, which motivates self on-policy distillation (OPD) to supply dense token-level supervision from a self-teacher with privileged task skills, letting a skill-free student internalize them. This recipe, however, is undermined by two findings in agentic tasks: privileged information alone does not always make a teacher reliable, and the benefit of teacher supervision is stage-dependent. We therefore propose RetireOPD (Self-Retiring On-Policy Distillation), which first optimizes a decoupled, ski

RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning

相关 · 按语义匹配