论文

研究表明在希腊语专业语料上，无参数 BM25 基线优于多个现成的多语言稠密检索模型，并推出首个大规模希腊语 RAG 基准 HERA，同时发布适配模型与基准以支撑未来希腊语 RAG 系统研究。This study shows that a parameter-free BM25 baseline outperforms several off-the-shelf multilingual dense retrieval models on specialist Greek corpora and introduces HERA, the first large-scale Greek benchmark for retrieval-augmented generation, and releases adapted models and benchmark to support future research on Greek-language RAG systems.

提出 AVE-Agent，一种模块化 agent 框架，将复杂指令分解为相互依赖的子任务，并通过自我反思与评估器反馈迭代改进编辑结果，在联合编辑中提升指令执行、保真度保持以及音视频对齐，同时保持有竞争力的感知质量。AVE-Agent is proposed, a modular agent framework that decomposes complex instructions into dependent subtasks and iteratively improves editing results through self-reflection and evaluator feedback, and improves instruction execution, Fidelity Preserving, and audio-visual alignment in joint editing while maintaining competitive perceptual quality.

提出 GDPevo，一种基于 GDP 相关企业工作流的 evolution-native 基准，并配套完全自动化的数据生成流水线；表现最佳的进化 agent 仍远低于全信息 oracle 上限，表明当前 agent 的自进化能力远未充分实现。GDPevo is presented, an evolution-native benchmark grounded in GDP-related enterprise workflows, together with the fully automated data pipeline that generates it, and the best evolved agents remain far below the fully informed oracle ceiling, indicating that the self-evolution ability of current agents remains far from fully realized.

这些结果将视觉地理定位确立为场景文本仲裁的连续诊断手段，并提供了一个受控框架，用于评估 MLLMs 如何解决冲突的多模态证据。These results establish visual geolocation as a continuous diagnostic of scene-text arbitration and provide a controlled framework for evaluating how MLLMs resolve conflicting multimodal evidence.

FinanceHarness 是一个运行金融工具与从业者引导工作流的框架，端到端自动化金融深度研究：环境与数据构建、Agent 执行循环以及奖励建模；FinanceGym 包含论点驱动的研报问题与评分标准，结合 pre-cutoff 与 post-cutoff 准则。FinanceHarness is presented, a harness that runs finance-oriented tools and practitioner-guided workflows, automating financial deep research end to end: environment and data construction, the agent execution loop, and reward modeling, and FinanceGym, comprising thesis-driven research questions and rubrics that combine pre-cutoff and post-cutoff criteria.

MameLoshnLM 是首个专为意第绪语构建的 8B 参数开源语言模型，既为意第绪语 NLP 提供基础，也为历史悠久但数字化程度不足的语言建模开发提供可复用的实践模板。MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish, is presented, providing both a foundation for Yiddish NLP and a practical template for language model development in historically rich but digitally underrepresented languages.

提出 DataSpace，一个基准，用于评估数据 Agent 在任务本地异构工作空间中生成可验证表格结果的能力，并指出提升数据 Agent 可靠性的关键挑战。DataSpace, a benchmark in which data agents produce verifiable tabular results from task-local heterogeneous workspaces, is introduced and key challenges for improving data-agent reliability are identified.

本文研究 11 项重大生活事件引发的人格变化，以大五人格作为心理测量锚点，并将所得轨迹与人类人格心理学的纵向证据进行对照，指出当前 PC-Agents 模拟了人类人格动态的均值，但未能模拟其形态。This work studies event-induced personality change after 11 major life events, using the Big Five traits as a psychometric anchor and interpreting the resulting trajectories against longitudinal evidence from human personality psychology, and suggests that current PC-Agents simulate the mean of human personality dynamics, but not its shape.

实验表明，对敏感信息的不必要获取广泛存在，并观察到任务完成能力与获取阶段泄露之间的相关性，因此对获取阶段隐私进行审计既紧迫也必要。The experiments show that the unnecessary acquisition of sensitive information is widespread, and a correlation between the task-completion capability and acquisition-stage leakage, and auditing acquisition-stage privacy both urgent and necessary is observed.

通过分别评估论文检索、证据 grounding 与答案准确性，LitTraceQA 为生成可验证答案、而非无依据摘要的科学 QA 系统提供了测试基准。By evaluating paper retrieval, evidence grounding, and answer accuracy separately, LitTraceQA provides a testbed for scientific QA systems that produce verifiable answers rather than unsupported summaries.

本文介绍了 CodeXGLUE，一个基准数据集，旨在推动面向程序理解与生成的机器学习研究，涵盖 14 个数据集上的 10 项任务，并提供模型评估与比较的平台。This paper introduces CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation that includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison.

于 2018 年 8 月 28 日中午 12:15 在 Pettit 微电子研究中心 102 A/B 室进行报告。Presented on August 28, 2018 at 12:15 p.m. in the Pettit Microelectronics Research Center, Room 102 A/B.

提出 FaceVid-Forensics-100K，一个大规模深伪视频数据集，包含 100,000 个视频，涵盖 33 种合成方法，覆盖换脸、表情重演与全脸合成；同时提出一个多智能体取证推理框架，由四个领域专家 Agent 分别从四个角度独立分析伪造线索。FaceVid-Forensics-100K is introduced, a large-scale deepfake video dataset comprising 100,000 videos and spanning 33 synthesis methods across face swapping, face reenactment, and entire-face synthesis, and a multi-agent forensic reasoning framework that employs four specialized domain-expert agents to independently analyze forgery cues from four perspectives.

CLIP-CC-Bench 为长视频描述提供了一个实用的评估框架，填补了现有短片段和仅 QA 基准的空白，并通过评分者间一致性（inter-judge agreement）与 bootstrap 排序稳定性量化该协议的内可靠性。CLIP-CC-Bench provides a practical evaluation framework for long-form video description, filling a gap left by existing short-clip and QA-only benchmarks and quantifying the protocol's internal reliability through inter-judge agreement and bootstrap ranking stability.

本文提出 MatrAIx，一个面向异构用户的群体规模模拟用户评估基础设施，为使用多样化模拟人类用户评估 AI 系统和数字产品提供端到端支撑。MatrAIx is introduced, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users and provides an end-to-end infrastructure for evaluating AI systems and digital products with diverse simulated human users.

Evo-Bench 是首个跨 Search、Office 和 General Agent 领域评估模型内在 harness 演化能力的基准，并暴露了早期饱和等关键时序异常，同时证明所合成的 harness 是高度可迁移的推理结构，能持续提升多样化策略模型。Evo-Bench is the first benchmark designed to evaluate models'intrinsic harness-evolving capabilities across Search, Office, and General agent domains, and exposes critical temporal anomalies like early saturation, while demonstrating that the synthesized harnesses act as highly transferable reasoning structures, consistently boosting diverse policy models.

本文提出 WeClawArena，一个面向个人工作空间多参与方 owned-agent 协作的可审计基准与运行时沙盒，基于有界运行时证据审计攻击成功情况，支持任务分解失败、隐私泄露、证据投毒以及权限路径失效等问题的诊断。WeClawArena is introduced, an auditable benchmark and runtime sandbox for multi-party owned-agent collaboration over personal workspaces and audits attack success from bounded runtime evidence, supporting diagnosis of task breakdown, privacy leakage, poisoned evidence, and invalid authority paths.

本文提出 MMOOC，一个用于评估 MLLMs 拒答与鲁棒回答能力的大规模 benchmark，并引入 LLM-as-a-Judge 指标来衡量模型推理的正确性。This work presents MMOOC, a large-scale benchmark for evaluating refusal and robust answering abilities of MLLMs, and introduces an LLM-as-a-Judge metric to assess the correctness of model reasoning.

本文倡导源对比评估，并构建了 Cultivar——FLORES 的本地化子集，可用于特定 locale 的翻译评估；研究发现：MT 专用模型鲁棒性较差，少数模型可能对 FLORES 存在过拟合，且模型普遍更擅长翻译美国 locale 的内容，而非其他 locale，无论语种如何。This work advocates for source-contrastive evaluation and instantiates Cultivar, a localised subset of FLORES, which enables locale-specific translation evaluation and finds that MT-specialised models are less robust, a few models potentially overfit FLORES, and models tend to translate US content better than that of other locales, regardless of language.

本文提出 Business Arena——一个受控环境，AI agent 在其中经营跨境店铺，在长周期内向供应商采购并向买家销售，迈出了构建面向端到端商业 agent 的真实可信测试床的第一步。Business Arena, a controlled environment where an AI agent runs a cross-border shop, buying from suppliers and selling to buyers over a long horizon, is introduced, taking a first step toward a realistic and trustworthy testbed for evaluating end-to-end business agents.

面向策略性优化下的可测性设计指南：保留探针仅在不可枚举轴上保持有效性；门控必须衡量保留集性能，而非仅正确性；迁移率只有在附带每类失败机制评级时才可解读。Design guidance for measurement under strategic optimization is distill design guidance for measurement under strategic optimization: held-out probes retain validity only on non-enumerable axes; gates must measure held-out performance, not just correctness; and a transfer rate is interpretable only with per-failure mechanism grades.

本文实现了对时间序列数据集相似度方法的系统化、可复现比较，提供灵活的扩展性以添加自定义数据集、相似度方法及下游时间序列任务，并通过集成的时间序列数据集归约器对数据集级和序列级相似度方法进行一致评估。This work enables systematic and reproducible comparisons of time-series dataset similarity methods, flexible extensibility for users to add customized datasets, similarity methods, and downstream time-series tasks, and consistent evaluation of both dataset-level and series-level similarity methods through integrated time-series dataset reducers.

本文提出解码层禁忌 (Decoding-Level Taboo)，一种零提示的诊断式压力测试，在运行时直接干预 logit 空间，于词边界处动态遮蔽主要候选 token，强制模型进行迂回表达。Decoding-Level Taboo is introduced, a zero-prompt diagnostic stress test that intervenes directly in logit space at runtime, forcing models out of their nominal paths by dynamically masking primary candidate tokens at word boundaries, forcing machine circumlocution.

360CityArena 为真实感城市区域导航与空间推理提供了必要且具有挑战性的测试平台；基于 SOTA LMM 智能体的评估显示，即使是最强的模型 Gemini 2.5 Flash，其表现仍远低于人类水平。360CityArena provides a necessary and challenging testbed for photorealistic urban-district navigation and spatial reasoning, and evaluation using state-of-the-art LMM-based agents shows that even the strongest model, Gemini 2.5 Flash, performs far below human level.

本文将该挑战形式化为叙事承诺保持 (Narrative Commitment Preservation, NCP)，并提出 NCP-Bench：一个基于电影剧情梗概构建的包含 100 个叙事环境的基准，每个环境均提供可在玩家智能体与叙述者智能体交互过程中自动检查的结构化叙事规范。This work forms this challenge as Narrative Commitment Preservation (NCP), and introduces NCP-Bench, a benchmark of 100 narrative environments derived from movie synopses that each environment includes a structured narrative specification that can automatically check throughout the interaction between the player agent and the narrator agent.

本文提出进化式马尔可夫超图攻击（EMHA），这是一种黑盒策略，通过协调授权状态转移执行反馈驱动的环境演化，无需参数更新，并将 OpenART 确立为在复杂演化环境中研究 Agent 安全性的可扩展基础。This work proposes the Evolutionary Markov Hypergraph Attack (EMHA), a black-box policy that performs feedback-driven environment evolution by coordinating authorized state transitions without requiring parameter updates, and establishes OpenART as a scalable foundation for studying agent safety in complex, evolving environments.

H2R-Bench 提供了一个系统性诊断框架，用于评估视频世界模型能否跨越 human-to-robot 具身差距，并将人类操作观测转化为以机器人为中心的训练资源。H2R-Bench provides a systematic diagnostic framework for evaluating whether video world models can bridge the human-to-robot embodiment gap and convert human manipulation observations into robot-centric training resources.

基于 CPI-Bench 对主流图像编辑模型的评测结果显示，CPI-Bench 增强了模型间的性能区分度；排名分析表明 CPI-Bench 与 Arena Image Edit Leaderboard 的对齐度最高，与公开人类偏好排名具有更强的一致性。Evaluation results of mainstream image editing models based on CPI-Bench demonstrate that CPI-Bench enhances performance differentiation among models, and ranking analysis reveals that CPI-Bench achieves the highest alignment with the Arena Image Edit Leaderboard, indicating stronger consistency with public human preference rankings.

在对 1,000 只美股进行小时级收益预测时，我们观察到一种意外现象：预测结果近乎平坦，且截面相关性衡量的股票排序能力很差，我们将其称为"预测坍缩"。令人惊讶的是，在同一设置下对交易量进行预测时，该现象基本消失。我们在多种时序基础模型（TSFMs）、12 个深度学习预测模型以及 97 个公开基准配置中系统考察了这一现象，发现其与目标可预测性密切相关，并识别出背后的两类成因：低 p……When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation. We call this forecast collapse. Surprisingly, the phenomenon largely disappears when forecasting trading volume under the same setting. We investigate forecast collapse across time-series foundation models (TSFMs), twelve deep-learning forecasting models, and 97 public benchmark configurations, and find that it is closely tied to target predictability. We identify two distinct reasons behind it: low p

一个机器人过程评估工具包，将 rollout 视频转化为稠密进度曲线并衍生多项细粒度指标；引入 RoboPulse++ 用于评估过程奖励模型（PRM）的可靠性，为评测者提供更准确的测试平台。A toolkit for robot process assessment that turns rollout videos into dense progress curves and derives multiple fine metrics, and introduces RoboPulse++ to evaluate the reliability of process reward models (PRM), providing evaluators with a more accurate testing platform.

提出 MobileMem，一个面向设备端长期记忆研究的 benchmark 和框架，基于长达一年的移动端经验集合，使 agent 能够记忆过去、理解当下并适应未来。This work introduces MobileMem, a benchmark and framework for studying on-device long-term memory, grounded in a year-scale collection of mobile experiences, and enables agents to remember the past, understand the present, and adapt to the future.

展望 ALD/E-ImageMiner benchmark 如何指导未来科学图像挑战赛，以及 Bloom-informed 问题设计如何支持更深入的科学理解。A forward-looking perspective is presented on how the ALD/E-ImageMiner benchmark can guide future scientific-image challenges and how Bloom-informed question design can support deeper scientific understanding.

提出 UNMASK，一个全自动 pipeline，可在无需额外人工标注的情况下发现、因果验证并缓解文本分类器中的伪相关，并证明其发现与验证阶段可泛化至奖励模型的偏好数据。U N M ASK is presented, a fully automated pipeline that discovers, causally verifies, and mitigates spurious correlations in text classifiers without additional human annotation, and demonstrates that the discovery and validation stages generalize to reward model preference data.

发现面向推理的训练并未优先放大具有最高 Lift 的行为，这促使研究者采用过程级目标，以奖励经过校准且有依据的推理，而不仅仅是表面形式。It is found that reasoning-oriented training does not preferentially amplify the highest-Lift behaviors, motivating process-level objectives that reward calibrated and grounded reasoning rather than surface form alone.

提出 Apodex Discovery，一个通过 heavy-duty solver 构建和评估发现型 AI 的框架；该 solver 包含一个 foundation model、harness、工具和控制策略，用于执行长期的、有状态的、可验证的探索。This work introduces Apodex Discovery, a framework for building and evaluating discoverative AI through the heavy-duty solver, a system comprising a foundation model, harness, tools, and control policies that pursues extended, stateful, verifiable investigations.