实验表明,OpenComputer 的硬编码验证器比 LLM-as-judge 评估更贴合人类裁定,尤其当任务成败取决于细粒度应用状态时。Experiments show that OpenComputer's hard-coded verifiers align more closely with human adjudication than LLM-as-judge evaluation, especially when success depends on fine-grained application state.
论文
677 张论文卡片 · OA 绿色
本文提出 MAGE(Memory as Agent-Guided Exploration),一种将交互存储在分层状态树中的主动式执行状态管理器;实验显示,MAGE 将平均任务成功率较基线提升 7.8–20.4 个百分点,同时 token 消耗减少 55.1%。This work proposes MAGE (Memory as Agent-Guided Exploration), an active execution-state manager that stores interactions in a hierarchical state tree and shows that MAGE improves the average task success rate by 7.8--20.4 pp over baselines, while reducing token consumption by 55.1%.
结果表明,思维轨迹是推理任务的有效检索语料;将其转换为结构化、紧凑化或诊断式表征后,可释放出更强的增益。The results suggest that thinking traces are an effective retrieval corpus for reasoning tasks, and transforming them into structured, compact, or diagnostic representations unlocks even stronger gains.
本文提出 M$^3$Exam,一个以查询为中心、基于真实用户-Agent 交互构建的多模态对话记忆基准,涵盖跨模态定位与隐式信息推断等多维度评估。M$^3$Exam is introduced, a query-centric multimodal conversational memory benchmark built on realistic user-agent interaction, with multi-dimensional evaluation spanning cross-modal grounding and implicit information inference.
本文综述了 Islamic LLMs 与可信 Islamic AI 这一新兴领域,并论证了仅具备阿拉伯语流利度不足以支撑 Islamic AI,进而提出面向抗幻觉 Islamic AI 系统的研究议程。This survey reviews the emerging field of Islamic LLMs and trustworthy Islamic AI, and argues that fluency in Arabic is not sufficient for Islamic AI, with a research agenda for hallucination-resistant Islamic AI systems.
本文提出 SCAR(Semantic Continuity-Aware Retrieval),一种自适应检索策略,通过权衡查询-邻居相关性与结构连续性惩罚来选择性扩展相邻分块,由此得到近似尺度不变的决策规则,无需重新校准即可跨 embedding 模型迁移。SCAR (Semantic Continuity-Aware Retrieval), an adaptive retrieval policy that selectively expands neighboring chunks by weighing query-neighbor relevance against a structural continuity penalty, is proposed, yielding an approximately scale-invariant decision rule that transfers across embedding models without recalibration.
本文提出 V-RAGBench——一个由 ⟨query, evidence chunk, answer⟩ 三元组构成的基准,可对检索与生成进行忠实且解耦的评估;同时提出 CARVE,一种在多种配置下并行运行检索器、并通过分块自适应重排序为每个分块挑选最优配置的简易方法。V-RAGBench is introduced, a benchmark of $\langle$ query, evidence chunk, answer$\rangle$ triplets that enables faithful, decoupled evaluation of retrieval and generation, and CARVE, a simple method that runs parallel retrievers across configurations and employs chunk-adaptive reranking to identify the winning configuration for each chunk.
研究发现,recall@k 并非已部署 KB-VQA 的正确评价指标,且弥合差距需要 reader 侧介入;本文首次对多模态 KB-VQA 中 reader 侧位置依赖性进行了受控探查,设计了一种 gold-position 协议——在问题提示中仅改变 gold passage 所在的槽位。The findings indicate that recall@k is the wrong metric for deployed KB-VQA and that closing the gap requires reader-side intervention; the first controlled probe of reader-side position dependence in multimodal KB-VQA is designed, a gold-position protocol in which only the gold passage's prompt slot varies within question.
本文提出 PathRouter,一种面向 agentic GraphRAG 的路径感知训练框架,沿答案正确性与证据路径重叠度联合评估每条轨迹,归纳出四类轨迹并采用差异化 GRPO 优势缩放,抑制捷径式强化同时保留证据寻求行为。This work presents PathRouter, a path-aware training framework for agentic GraphRAG that jointly evaluates each trajectory along answer correctness and evidence-path overlap, yielding four trajectory categories with differentiated GRPO advantage scaling that suppresses shortcut reinforcement while preserving evidence-seeking behavior.
本文提出 User as Code(UaC)范式,将 Agent 对用户的建模视为一个活的软件项目:类型化 Python 对象承载用户状态,普通 Python 函数编码其治理规则,从而在解释器可运行的同一媒介内完成对用户的表示与推理。This work introduces User as Code (UaC), a paradigm in which an agent's model of a user is a living software project: typed Python objects hold the user's state and ordinary Python functions encode the rules that govern it, so representing and reasoning about the user happen in one medium an interpreter can run.
本文提出 Diverge,一种即插即用的 agentic RAG 框架,通过迭代式、反思引导的多视角探索以及多样性感知检索支持来改善多样性—质量权衡,并引入用于刻画开放域问答中多样性—质量权衡的评估指标。Diverge is proposed, a plug-and-play agentic RAG framework that improves the diversity--quality trade-off through iterative, reflection-guided exploration of diverse viewpoints and diversity-aware retrieval support, and introduces evaluation metrics for characterizing the diversity-quality trade-off in open-ended question answering.
本文命名并研究这两条研究脉络之间的模式:其递归单元是配备文件系统工具、代码执行与规划的完整 Agent harness,而非无工具的模型调用,并给出针对长上下文推理的受控评估。This work names and studies the pattern between these two lines of work, where the recursive unit is a full agent harness with filesystem tools, code execution, and planning rather than a model call with no tools, and provides a controlled evaluation on long-context reasoning.
作者的分析揭示了基于扩展设计的基本局限:扁平化层级会导致 PE-Online 中递归查询延迟过高,并在两种扩展策略下产生结构变更时不可扩展的写放大;与之相对,TrieHI 将目录拓扑保留为原生前缀树,通过树遍历实现高效递归检索,借助拓扑节点操作降低维护成本。The authors' analysis exposes the fundamental limitations of expansion-based designs: flattening the hierarchy incurs high recursive-query latency in PE-Online and unscalable write amplification during structural changes in both expansion strategies, and in contrast, TrieHI keeps the directory topology as a native prefix tree, enabling efficient recursive retrieval through tree traversal and reducing maintenance cost through topological node manipulation.
本文提出 ForeSci,一个时间受控的基准,用于评估 LLM Agent 是否能从历史证据中做出前瞻性研究判断,并在四种骨干模型上评测原生 LLM、Hybrid RAG 以及三种 research-agent 适配方案。This work introduces ForeSci, a temporally controlled benchmark for evaluating whether LLM agents can make such forward-looking research judgements from historical evidence, and evaluates native LLMs, Hybrid RAG, and three research-agent adaptations across four backbones.
本文提出名为 InSemRAG 的 RAG 框架,通过迭代检索—校验机制及两个支撑模块——意图感知检索器(IAR)与语义保持切分(SPC)——应对上述挑战。This work proposes a RAG framework, termed InSemRAG, that addresses these challenges via an iterative retrieve-and-check mechanism with two supporting modules, an intention-aware retriever (IAR) and semantics-preserving chunking (SPC).
本文提出 π-Bench,一个用于评估主动式协助能力的基准,包含跨 5 个领域特定用户画像的 100 个多轮任务,用于评估 Agent 在长交互中预见并满足用户需求的能力,联合衡量长周期轨迹中的主动性与任务完成度,更贴近真实使用场景。$-Bench is introduced, a benchmark for proactive assistance comprising 100 multi-turn tasks across 5 domain-specific user personas that evaluates agents'ability to anticipate and address user needs over extended interactions, jointly measuring proactivity and task completion in long-horizon trajectories that better reflect real-world use.
MRAgent,一种将联想记忆图与主动重构机制相结合的框架,将 LLM 推理直接融入记忆访问,确保记忆检索能动态适配推理上下文,同时避免无约束扩展引发的组合爆炸。MRAgent, a framework that combines an associative memory graph with an active reconstruction mechanism that integrates LLM reasoning directly into memory access, ensuring that memory retrieval is dynamically adapted to the reasoning context while avoiding combinatorial explosion caused by unconstrained expansion.
本文提出 Vision-aligned Latent Reasoning(VaLR),一种简洁而有效的推理框架,在每个 Chain of Thought 推理步骤之前动态生成视觉对齐的 latent token,引导模型在 latent space 中基于感知线索进行推理。Vision-aligned Latent Reasoning (VaLR) is introduced, a simple, yet effective reasoning framework that dynamically generates vision-aligned latent tokens before each Chain of Thought reasoning step, guiding the model to reason based on perceptual cues in the latent space.
本文提出 ContextRL,一种上下文感知的强化学习方法,通过间接辅助目标提升长周期推理与多模态性能,并与将相同对比上下文复用作标准 query–context–answer 样本的数据增强基线进行对比。This work proposes ContextRL, a context-aware reinforcement learning (RL) method that improves long-horizon reasoning and multimodal performance through an indirect auxiliary objective, and compares against data-augmentation baselines that repurpose the same contrastive contexts as standard query--context--answer examples.
本文提出 MMLongEmbed,首个面向长上下文场景评估 MEM 的综合基准,并发现现有架构严重依赖浅层特征匹配,难以捕捉深层语义与结构依赖。This work introduces MMLongEmbed, the first comprehensive benchmark for evaluating MEMs in long-context scenarios, and finds that current architectures rely heavily on superficial feature matching and struggle to capture deep semantic and structural dependencies.
Agents' Last Exam(ALE)是一个面向长时序、具有经济价值且结果可验证的真实任务的 AI Agent 评测基准,旨在弥合基准测试表现与 GDP 相关影响之间的差距,而非仅仅作为排行榜。Agents'Last Exam (ALE) is introduced, a benchmark designed to evaluate AI agents on long horizon, economically valuable, real world tasks with verifiable outcomes, intended not merely as another leaderboard, but as an instrument for closing the gap between benchmark success and GDP relevant impact.
AsymCache 是一个面向 LLM 推理的计算-延迟感知 KV cache 管理系统,将 cache 驻留决策与 GPU attention kernel 性能显式对齐,包含三个关键组件:用于高效处理非连续 KV 上下文的多段注意力(MSA)、联合优化命中率与位置感知重计算代价的 cache 淘汰策略,以及面向高硬件利用率的自适应分片调度器。AsymCache is proposed, a computation-latency-aware KV cache management system for LLM inference that explicitly aligns cache residency decisions with GPU attention kernel performance, including three key components: Multi-Segment Attention (MSA) for efficient non-contiguous KV context processing, a cache eviction policy that jointly optimizes hit rate and position-aware recomputation cost, and an adaptive chunking scheduler for high hardware utilization.
提出一个面向 RAG 的 AI 系统基准测试(RAGPerf)框架,用于刻画 RAG pipeline 的系统行为,并证明其引入的性能开销可忽略不计。The design and implementation of a RAG-based AI system benchmarking (RAGPerf) framework for characterizing the system behaviors of RAG pipelines is presented and it is shown that RAGPerf incurs negligible performance overhead.
RTP-LLM 是一个面向工业级 LLM 部署的高性能推理引擎,已在 Alibaba Group 成功部署,服务超过 1 亿用户,通过集成设计解决根本性瓶颈。RTP-LLM is presented, a high-performance inference engine for industrial-scale LLM deployment, successfully deployed across Alibaba Group serving over 100 million users, and addresses fundamental bottlenecks through integrated design.
这篇立场论文认为,LLM 推理 serving 已超越通用启发式方法,如今需要数学优化与算法基础,并呼吁社区将 LLM serving 的算法设计视为一个新的研究前沿。This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic foundations, and calls on the community to recognize algorithmic design for LLM serving as a research frontier.
本文首次系统分析了 LLM 如何通过强化学习(RL)学习针对不同 retriever 调整 query 表述策略,并揭示 RL 能有效教会 LLM 根据特定 retriever 特性定制 query。This work presents the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL), and reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics.
SwiftCache 是一个协同推理系统,使异构模型可在同一服务器内共享未充分利用的 GPU 内存与 NVLink 带宽,支持跨模型通过 NVLink 共享 KV cache,避免使用慢速 PCIe 传输。SwiftCache is a collaborative inference system that enables heterogeneous models to share underutilized GPU memory and NVLink bandwidth within a server, allowing cross-model KV cache sharing over NVLink and avoiding slow PCIe transfers.
一个两级框架将手动 harness 工程转变为自动化 harness 工程,并更进一步——将"自动化本身的设计"也自动化。A two-level framework shifts manual harness engineering into automated harness engineering, and takes one step further --automating the design of the automation itself.
综合结果表明,强大的 coding agent 通过使用工具、反馈和工作区状态来构建目标语言的可工作模型,从而适应不熟悉的语言。Together, these results show that strong coding agents adapt to unfamiliar languages by using tools, feedback, and workspace state to build a working model of the target language.
提出 Agentic Harness Engineering(AHE),一个通过三个相互匹配的 observability 支柱应对 harness 工程挑战的闭环,将每一次编辑转化为可证伪的契约,使 harness 演进能够自主进行而不退化为试错。Agentic Harness Engineering (AHE) is introduced, a closed loop that addresses harness engineering challenges through three matched observability pillars that turn every edit into a falsifiable contract, so harness evolution proceeds autonomously without collapsing into trial-and-error.
提出 OScaR(Omni-Scaled Canalized Rotation),一个面向 X-LLMs 的精确且轻量的 KV cache 压缩框架,作为一个鲁棒、低复杂度、通用的框架确立了新的 Pareto 前沿。OScaR (Omni-Scaled Canalized Rotation), an accurate and lightweight KV cache compression framework for X-LLMs, is proposed, establishing it as a robust, low-complexity, and universal framework that defines a new Pareto front.
本文提出 Natural-Language Agent Harnesses,即可编辑的、描述运行级 harness 策略的文档,以及 Intelligent Harness Runtime(IHR),一个将上述文档解释为 agent 调用、交接、状态更新、验证门控与 artifact 契约的共享运行时。This paper introduces Natural-Language Agent Harnesses, editable documents that describe run-level harness policy, and Intelligent Harness Runtime (IHR), a shared runtime that interprets these documents into agent calls, handoffs, state updates, validation gates, and artifact contracts.
提出 agent harness 的操作性定义,配套统一词汇,可用于指导工程实践以及对 agentic 系统的科学比较。An operational definition of agent harness is proposed, with a shared vocabulary, able to guide engineering practice and the scientific comparison of agentic systems.
WAIT (Waiting for Accumulated Inference Threshold) 是一种基于阈值的准入规则,适用于已知输出长度;Nested WAIT 通过调控请求在 decode 阶段各分段间的推进方式,将该规则扩展到未知输出长度场景。WAIT (Waiting for Accumulated Inference Threshold), a threshold-based admission rule for known output lengths, and Nested WAIT, which extends the rule to unknown output lengths by regulating how requests advance across decode-stage segments are designed.
本文旨在应对常见挑战并简化 RAG 与既有企业基础设施的集成,提出一套面向可扩展本地化部署企业 RAG 方案的综合性 AI 工程蓝图。This paper aims to address the gap in comprehensive on-premises RAG implementation frameworks by presenting a comprehensive AI engineering blueprint for scalable on-premises enterprise RAG solutions to address common challenges and streamline the integration of RAG into existing enterprise infrastructure.
本文对近期 KV cache 优化技术进行系统综述,将其归纳为五大方向:cache eviction、cache compression、混合内存方案、新型 attention 机制与组合策略,并指出自适应多阶段优化流水线是未来研究的重要方向。This paper provides a systematic review of recent KV cache optimization techniques, organizing them into five principal directions: cache eviction, cache compression, hybrid memory solutions, novel attention mechanisms, and combination strategies, and pointing toward adaptive, multi-stage optimization pipelines as a promising direction for future research.