内容库 / 论文

arXiv:2608.13622 · Agent 智能体

ARC: Fair Relative Advantage Comparison in Open-Ended Real-World Interaction

ARC：开放式真实交互中的公平相对优势比较

ARC: Fair Relative Advantage Comparison in Open-Ended Real-World Interaction

类型：arxiv
标识：2608.13622
链接：https://arxiv.org/abs/2608.13622
主分类：agent
形态：method
TLDR：Open-ended real-world interaction admits multiple valid behaviors: an agent may answer directly, ask for clarification, provide progress updates, or confirm before acting. This flexibility breaks a core assumption behind group-based RL: rollouts compared within a group are no longer guaranteed to be behaviorally comparable. As a result, reward-model preferences over interaction style can distort relative advantages and steer optimization toward reward-preferred behaviors rather than context-appropriate ones. We formalize this as a reward fairness problem and propose ARC (Advantage Regularizati
待LLM分类：否
标题中文：ARC：开放式真实交互中的公平相对优势比较
TLDR中文：开放式真实交互允许多种有效行为：Agent 可直接回答、请求澄清、提供进度更新，或在执行前进行确认。这种灵活性打破了基于分组的 RL 的核心假设：同一分组内对比的 rollout 不再保证行为可比。因此奖励模型对交互风格的偏好可能扭曲相对优势，使优化偏向奖励偏好的行为而非情境适配的行为。我们将其形式化为奖励公平性问题，并提出 ARC（Advantage Regularizati
来源文件：
/inbox/tom/_candidates/2026-08-25-agent-rag-longcontext-candidates.json

相关 · 按语义匹配

笔记精读 · Perception-R1（Visual Perception Reward for MLLM RLVR）

笔记 flyP 精读与反方审稿 · Agentic RL 综述 + GLM-5.2 开放 Agent 路线 · 2026-07-15 22:50

笔记 Agent-STAR / arXiv:2603.21972 · "Demystifying RL for Long-Horizon Tool-Using Agents: A Comprehensive Recipe" · flyP 精读与批判

笔记 ReOPD (arXiv:2607.04763v2) 精读与批判 — flyP

笔记 ContextRL: Context-Aware RL for Agentic and Multimodal LLMs

笔记 ContextRL:Context-Aware 强化学习用于 Agentic 与多模态 LLM(精读 · flyP)

arXiv

2608.13622

TLDR

开放式真实交互允许多种有效行为：Agent 可直接回答、请求澄清、提供进度更新，或在执行前进行确认。这种灵活性打破了基于分组的 RL 的核心假设：同一分组内对比的 rollout 不再保证行为可比。因此奖励模型对交互风格的偏好可能扭曲相对优势，使优化偏向奖励偏好的行为而非情境适配的行为。我们将其形式化为奖励公平性问题，并提出 ARC（Advantage RegularizatiOpen-ended real-world interaction admits multiple valid behaviors: an agent may answer directly, ask for clarification, provide progress updates, or confirm before acting. This flexibility breaks a core assumption behind group-based RL: rollouts compared within a group are no longer guaranteed to be behaviorally comparable. As a result, reward-model preferences over interaction style can distort relative advantages and steer optimization toward reward-preferred behaviors rather than context-appropriate ones. We formalize this as a reward fairness problem and propose ARC (Advantage Regularizati

相关解读

arXiv ↗ 中文全文