论文

本文提出 π-Bench，一个用于评估主动式协助能力的基准，包含跨 5 个领域特定用户画像的 100 个多轮任务，用于评估 Agent 在长交互中预见并满足用户需求的能力，联合衡量长周期轨迹中的主动性与任务完成度，更贴近真实使用场景。$-Bench is introduced, a benchmark for proactive assistance comprising 100 multi-turn tasks across 5 domain-specific user personas that evaluates agents'ability to anticipate and address user needs over extended interactions, jointly measuring proactivity and task completion in long-horizon trajectories that better reflect real-world use.

Agents' Last Exam（ALE）是一个面向长时序、具有经济价值且结果可验证的真实任务的 AI Agent 评测基准，旨在弥合基准测试表现与 GDP 相关影响之间的差距，而非仅仅作为排行榜。Agents'Last Exam (ALE) is introduced, a benchmark designed to evaluate AI agents on long horizon, economically valuable, real world tasks with verifiable outcomes, intended not merely as another leaderboard, but as an instrument for closing the gap between benchmark success and GDP relevant impact.

本文构建一种针对 GitHub issue 的持久会话评估方法，锚定在单一 base commit，对线性顺序探索与非线性、领域范围的并行 agentic 探索进行比较。This work constructs an approach for persistent-session evaluation of GitHub issues anchored at a single base commit, and compares linear sequential exploration against non-linear, domain-scoped parallel agentic exploration.

结果表明，在所评估的协调者–工作者设定下，多 Agent 系统中的隐私风险主要由架构层面的协调通道决定，而非仅取决于最终输出行为：风险来源于对标准输出级防御不可见的内部通道。Results suggest, within the evaluated coordinator-worker setting, that privacy risk in multi-agent systems is strongly shaped by architectural coordination channels rather than final-output behavior alone: it arises from internal channels that remain invisible to standard output-level defenses.

介绍 MCP-Persona，这是首个专为评估 Agent 在真实场景、个性化 MCP 工具上的表现而设计的基准，并揭示了当前 Agent 在个性化工具使用上的显著不足，从而凸显该基准在发现并解决这些局限上的关键作用。MCP-Persona is introduced, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools and demonstrates their significant struggles with personalized tool use, thereby highlighting the benchmark's crucial role in identifying and addressing these limitations.

本文提出 AgenticRAGTracer，这是首个主要由大语言模型自动构建、专为支持逐步验证而设计的 Agentic RAG 基准。AgenticRAGTracer is introduced, the first Agentic RAG benchmark that is primarily constructed automatically by large language models and designed to support step-by-step validation, and is primarily constructed automatically by large language models and designed to support step-by-step validation.

本工作将带外防御组织为经典完整性保护、引用监控与最小权限的具体实例，对它们覆盖与未覆盖的内容进行结构化对比；与该假设一致但尚未被证实的是：确定性的带外强制执行相比带内检测，是更难被自适应攻击者攻破的目标。This work organizes out-of-band defenses as instances of classical integrity protection, reference monitoring, and least privilege, yielding a structured comparison of what they do and do not cover, consistent with, but not established, the hypothesis that deterministic out-of-band enforcement is a harder target for an adaptive attacker than in-band detection.

本文展示了用于 IT-Grundschutz（IT-GS）认证部分自动化的多 Agent 系统（MAS）架构结合混合检索增强生成（HybridRAG）的技术实现与实证评估，并为强化合规严谨性引入两项新的 MAS 架构技术贡献。This paper presents the technical implementation and empirical evaluation of a Multi-Agent System (MAS) architecture combined with Hybrid Retrieval Augmented Generation (HybridRAG) for the partial automation of IT-GS certification and introduces two novel technical contributions to the MAS architecture to enforce the compliance rigor.

结果表明，工具使用评测应从函数调用准确率转向不可靠工具环境下的任务完成度，并建议工具使用评测应从函数调用准确率转向不可靠工具环境下的任务完成度。（注：原文末句疑似重复）Results suggest that tool-use evaluation should move beyond function-call accuracy toward task completion under unreliable tool environments, and suggest that tool-use evaluation should move beyond function-call accuracy toward task completion under unreliable tool environments.

提出 GateMem，一个面向多主体共享内存 Agent 的基准，联合评估合法长程请求及其状态更新的效用、跨上下文授权边界的访问控制，以及 Agent 在收到显式删除请求后的主动遗忘能力。GateMem is introduced, a benchmark for multi-principal shared-memory agents that jointly evaluates utility for legitimate long-horizon requests with state updates, access control across contextual authorization boundaries, and agent-facing active forgetting after explicit deletion requests.

对涵盖前沿闭源与开源模型、共 22 个模型在多个规模上的评估发现，所有模型家族均存在不可忽略的隐私泄露，且指令遵循能力与泄露率呈正相关。Evaluating 22 models spanning frontier proprietary and open-source models at multiple scales, it is found that all model families exhibit non-trivial leakage, and that instruction- following ability correlates with leakage rate.

结果表明模型倾向于选择有害场景，在中性预订选项上的表现低于随机猜测水平，其中 Claude 4.8 取得最高分 64.7%。The results indicate that models tend to prefer harmful scenarios, performing below the random chance rate for selecting a neutral booking option, with Claude $4.8$ achieving the highest performance at $64.7\%$.

新加坡 AI Safety Institute 与韩国 AI Safety Institute 联合评估了涵盖客服、DevOps、网页自动化以及企业与个人生产力场景下 12 项真实非对抗任务中的 Agent 数据泄露问题，表明操作性数据泄露是与对抗性数据外泄不同的一阶 Agent 安全问题。A joint evaluation by the Singapore AI Safety Institute and the Korea AI Safety Institute examining agent data leakage in 12 realistic, non-adversarial tasks spanning customer support, DevOps, web automation, and enterprise and personal productivity indicates that operational data leakage is a first-order agent-safety concern distinct from adversarial exfiltration.

对 Agent 行为的分析揭示了长视野经济交互中的显著差异：表现更好的模型与其他企业的沟通更为活跃，而 Claude Haiku 4.5 则表现出 idle-drift 失效模式，在生成连贯评估与规划的同时仍反复选择不行动。Analysis of agent behavior reveals substantial differences in long-horizon economic interaction: higher-performing models communicate more actively with other firms, whereas Claude~Haiku~4.5 exhibits an idle-drift failure mode, repeatedly choosing inaction despite producing coherent assessments and plans.

本文提出了一种动作分级伤害评估量表，按七级有序量表对智能体的工具调用轨迹进行打分，评判依据包括所执行动作的可逆性、是否越界涉及其他方以及是否扩大了权限。An action-graded harm rubric is introduced that scores an agent's tool-call trajectory on a seven-level ordinal scale according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege.

提出 AgentLens，一个面向交互式代码 Agent 的生产级评估基准，将形式化验证（在存在客观检查时）与 LLM 编写的轨迹评审及并排比较相结合，使每次运行都能给出关于分数为何如此的可读解释。This work presents AgentLens, a production-assessed benchmark for interactive code agents that pairs formal verification, where an objective check exists, with LLM-written trajectory reviews and side-by-side comparisons, so that each run yields a readable explanation of why the score is what it is.

提出首个面向动态真实世界场景评估主动 agent 的能力驱动基准，并在模型与框架层面的全面比较表明，基模型能力与 agent 框架设计共同决定了真实世界环境中的性能表现。The first capability-driven benchmark designed to evaluate proactive agents in dynamic, real-world settings is introduced, and comprehensive comparisons across both models and frameworks show how base model capabilities and agent framework designs jointly shape performance in real-world environments.

该工作提出 LongMedBench，一个基于真实 EHR 的长程临床决策基准，并设计了一套包含三个评估维度的分类体系：事实型问答、时序推理、长程决策。This work introduces LongMedBench, a real-world EHR-based benchmark for long-horizon clinical decision-making, and proposes an evaluation taxonomy with three suites: fact-based QA, temporal reasoning, and long-horizon decision-making.

该工作提出 Long-Horizon-Terminal-Bench，一个涵盖九大类共 46 个长程任务的终端基准，包括实验复现、软件工程、多模态分析、交互游戏与科学计算，并分析失败模式与错误规律，以推动长程终端 Agent 的后续研究。This work introduces Long-Horizon-Terminal-Bench, a terminal benchmark of 46 long-horizon tasks spanning nine categories, including experiment reproduction, software engineering, multimodal analysis, interactive games, and scientific computing, and analyzes failure modes and error patterns to support future progress on long-horizon terminal agents.

论文提出 Flow-ERD，一个同时追求真实性与多样性的多 Agent 仿真器，在 WOSAC 测试基准上排名第一，并在可复现基线中主导真实性-多样性 Pareto 前沿。Flow-ERD is introduced, a multi-agent simulator that pursues realism and diversity jointly and ranks first on the WOSAC test benchmark and dominates the realism--diversity Pareto front among reproducible baselines.

实验表明，EvolvingWorld 能够通过有效维持持久且一致的角色与世界发展，提升长程模拟能力。Experiments show that EvolvingWorld can improve long-horizon simulation by effectively maintaining persistent, coherent character and world development.

本文提出 Manager Coercion Benchmark：被测 manager 拥有一个良性任务并有完成动机，但唯一能够礼貌且坚定拒绝执行任务的 agent，正是被测 manager 本身。The Manager Coercion Benchmark is introduced: the manager under test needs a benign task done and has an incentive to deliver, but the only agent that can do it politely and immovably declines, but the only agent that can do it politely and immovably declines is the manager under test.

本文是面向业务流程中长时间运行、有状态、多步生成式 AI 系统的基于图的工作流路径实践指南，并通过三个可执行示例展示类型化状态、条件路由、确定性工具、重试、中断、检查点与 trace 如何协同工作。This paper is a practitioner guide to graph-based workflow pathways for long-running, stateful, multi-step generative AI systems in business processes and presents three executable recipes to show how typed state, conditional routing, deterministic tools, retries, interrupts, checkpoints, and traces fit together.

本文提出 RF-Agent，通过多 Agent 的 Question-Thinking-Solution-Answer 流水线，基于教材驱动的知识蒸馏来弥补 RF 领域专用推理的空白，为面向 LLM 辅助 RF 电路设计的未来工作提供了可复用的基础。RF-Agent is presented, which addresses the gap in domain-specific RF reasoning through textbook-driven knowledge distillation through a multi-agent Question-Thinking-Solution-Answer pipeline and provides a reusable foundation for future work on LLM-aided RF circuit design.

EduPanel 是一个基于评分量表、以学习者为条件的 LLM 评判器，通过在多个专用 agent 间分解评估流程，对教学质量的各个方面产出可解释的评估结果，其可靠性与中等水平的人类专家相当。E EduPanel, a rubric-grounded, learner-conditioned LLM judge that decomposes evaluation across specialized agents to produce interpretable assessments for different aspects of teaching quality, achieves reliability comparable to a median human expert.

介绍 HANDBOOK_md，一个包含 65 个 agentic 任务的 benchmark，模拟员工遵循公司手册的方式；每项任务对 10 份基础手册之一进行修改，变动评分所依赖的具体规则和阈值，因此没有任何两个任务共享同一套策略。HANDBOOK_md is presented, a benchmark of 65 agentic tasks modeled on how employees follow company handbooks, and every task modifies one of 10 base handbooks, altering the specific rules and thresholds on which grading depends, so no two tasks share the same set of policies.

一项受控的种子干预实验发现，与随机非 gold 上下文相比，由检索得到的初始上下文以更少的后续探索取得更高的文件 F1，而 oracle gold 上下文仍存在可观的提升空间。A controlled seed-intervention pilot finds that retrieval-derived initial context yields higher file F1 with less post-seed exploration than random non-gold context, while oracle gold context shows substantial remaining headroom.

介绍 StealthBench，一个跨越六个 OPSEC 维度衡量自主攻击性安全 agent 操作隐蔽性的 benchmark，并以公共 benchmark 形式发布，以支持隐蔽感知 agent 的开发及自主攻击性安全部署中的自动化 OPSEC 监控。StealthBench, a benchmark that measures operational stealth in autonomous offensive-security agents across six operational security (OPSEC) dimensions, is introduced and released as a public benchmark to support both the development of stealth-aware agents and automated OPSEC monitoring for autonomous offensive-security deployments.

提出 SpecFirst，一个两阶段框架，在代码合成前强制进行需求 elicitation，并证明显式需求工程阶段是从零构建程序的有效范式。This work presents SpecFirst, a two-stage framework that forces the specification elicitation before code synthesis, and demonstrates that an explicit requirements-engineering phase is an effective paradigm for from-scratch program construction.

教育agentic工程师需要系统性变革而非增量式课程改革：教学必须从产出工件转向对日益自主的社会-技术系统进行判断。It is concluded that educating the agentic engineer requires systemic transformation rather than incremental curricular change: instruction must shift from producing artifacts to exercising judgment over increasingly autonomous socio-technical systems.