ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
- 类型：arxiv
- 标识：2609.35215
- 链接：http://arxiv.org/abs/2609.35215v1
- 主分类：agent
- 形态：application
- TLDR：Terminal utility evaluates a complete agentic workflow, but learning requires credit for the decisions within it. We introduce Attentive Search over Counterfactual Trees (ASCT), a framework that turns training-time multi-step search into local action credit. At actor-visited states, an auxiliary tree evaluates alternative legal actions from the same recoverable prefix. Its action-value table is centered by the frozen actor's probabilities and supplies credit for PPO on actor-sampled trajectories. This protocol connects counterfactual evaluation to policy learning while deploying the actor alon
- 副分类：evaluation
- 待LLM分类：否
- 标题中文：ASCT: 在反事实树上进行注意力搜索，用于 Agentic 强化学习中的信用分配
- TLDR中文：终端效用评估完整的 Agentic workflow，而学习则需要对其中的决策进行信用分配。我们提出在反事实树上进行注意力搜索（ASCT），一个将训练时的多步搜索转化为局部动作信用的框架。在 actor 访问到的状态处，一棵辅助树从同一可恢复前缀评估其他合法动作。其动作价值表以冻结 actor 的概率为中心，为 actor 采样轨迹上的 PPO 提供信用。该协议在部署 actor 的同时，将反事实评估与策略学习连接起来。
- 来源文件：
- /inbox/tom/_candidates/2026-09-30-agent-rag-longcontext-candidates.json