Enhancing Rubric-based RL via Self-Distillation

• 类型：arxiv
• 标识：2607.18082
• 链接：https://arxiv.org/abs/2607.18082
• 主分类：llm-infra
• 形态：method
• 被引：1
• 被引来源：Semantic Scholar
• S2被引：1
• OpenAlex被引：0
• 影响力被引：0
• TLDR：Criterion-Distilled Policy Optimization (CriPO) is proposed, which enhances rubric-based RL via on-policy self-distillation and employs a counterfactual self-teacher to locate criterion-relevant tokens in negative-advantage rollouts and flips their token-level advantages to positive values, preserving useful patterns that would otherwise be suppressed.
• OpenAlex ID：W7169884067
• OpenAlex DOI：10.48550/arxiv.2607.18082
• DOI：10.48550/arxiv.2607.18082
• DOI来源：OpenAlex
• 开放获取：green
• 开放获取链接：https://doi.org/10.48550/arxiv.2607.18082
• OpenAlex更新：2026-08-26
• 待LLM分类：否
• 标题中文：通过自蒸馏增强基于评分标准的强化学习
• TLDR中文：提出Criterion-Distilled Policy Optimization (CriPO)，通过on-policy自蒸馏增强基于rubric的RL，并采用反事实自教师定位负优势rollout中与准则相关的token，将其token级优势翻转为正值，保留本将被抑制的有用模式。
• 来源文件：
• /inbox/tom/_candidates/2026-08-03-agent-rag-longcontext-candidates.json
• [S2 enrich]
• [OpenAlex backfill]