研究库 论文知识库

内容库 / 论文

arXiv:2609.12419 · 工程化

MInTRL: Off-policy Intervention can boost On-policy RL

MInTRL：Off-policy 干预可增强 On-policy RL

MInTRL: Off-policy Intervention can boost On-policy RL

类型：arxiv
标识：2609.12419
链接：https://arxiv.org/abs/2609.12419
主分类：engineering
形态：method
TLDR：Reinforcement learning with verifiable rewards is typically performed on-policy, keeping training data close to the current policy but limiting learning to trajectories that the policy can discover itself. Off-policy methods such as supervised fine-tuning, on the other hand, can leverage external knowledge beyond the base model's capabilities, but may suffer from large distribution shift. The key challenge is thus to expand exploration without sacrificing learnability. In this work, we introduce Minimal Intervention Reinforcement Learning (MInTRL), which expands the exploration frontier throug
待LLM分类：否
标题中文：MInTRL：Off-policy 干预可增强 On-policy RL
TLDR中文：带可验证奖励的强化学习通常以 on-policy 方式进行，训练数据保持接近当前策略，但学习仅限于策略自身可发现的轨迹。Off-policy 方法如监督微调则可利用基础模型能力之外的外部知识，但可能存在较大的分布偏移。核心挑战在于扩展探索的同时不牺牲可学习性。本工作提出 Minimal Intervention Reinforcement Learning (MInTRL)，通过 throug
来源文件：
/inbox/tom/_candidates/2026-09-15-agent-rag-longcontext-candidates.json

相关 · 按语义匹配

笔记 flyP 精读与反方审稿 · Agentic RL 综述 + GLM-5.2 开放 Agent 路线 · 2026-07-15 22:50

笔记 Agent-STAR / arXiv:2603.21972 · "Demystifying RL for Long-Horizon Tool-Using Agents: A Comprehensive Recipe" · flyP 精读与批判

仓库 Dylsimple60/RLHF_learn

笔记精读与批判 · RAGEN-2:Reasoning Collapse in Agentic RL

笔记 ReOPD (arXiv:2607.04763v2) 精读与批判 — flyP

笔记 Cameron R. Wolfe (Deep Learning Focus) · RSS 摘要

arXiv

2609.12419

TLDR

带可验证奖励的强化学习通常以 on-policy 方式进行，训练数据保持接近当前策略，但学习仅限于策略自身可发现的轨迹。Off-policy 方法如监督微调则可利用基础模型能力之外的外部知识，但可能存在较大的分布偏移。核心挑战在于扩展探索的同时不牺牲可学习性。本工作提出 Minimal Intervention Reinforcement Learning (MInTRL)，通过 througReinforcement learning with verifiable rewards is typically performed on-policy, keeping training data close to the current policy but limiting learning to trajectories that the policy can discover itself. Off-policy methods such as supervised fine-tuning, on the other hand, can leverage external knowledge beyond the base model's capabilities, but may suffer from large distribution shift. The key challenge is thus to expand exploration without sacrificing learnability. In this work, we introduce Minimal Intervention Reinforcement Learning (MInTRL), which expands the exploration frontier throug

arXiv ↗ 中文全文