论文

CAPA通过六种机制刻画个性化编码歧义，并使用受控的三阶段生成流程将这些机制注入无歧义的可执行任务，为开发长期编码助手奠定基础，使其生成的代码更好地对齐用户意图并减少反复澄清。CAPA, which characterizes personalized coding ambiguity through six mechanisms and injects these mechanisms into unambiguous executable tasks using a controlled three-stage generation pipeline, provides a foundation for developing long-term coding assistants that better align generated code with user intent while reducing repeated clarification.

本文提出以交互为中心的分类法，将失败定位到其起源的交互并识别责任组件，将 41 种失败模式归到两个组件之间的边及指示修复归属的故障侧This work introduces an interaction-centric taxonomy that localizes failures to the interactions in which they originate and identifies the responsible component, and organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs.

提出 AntiSkillBench，一个端到端的基准，用于评估 persona-skill 流水线中的风险与防御；实验表明 persona-skill 风险在不同的 agent backbone 和蒸馏协议下持续存在，从显式属性扩展到沟通风格与个性特征。AntiSkillBench is introduced, an end-to-end benchmark for evaluating risks and defenses across the persona-skill pipeline, and experiments show that persona-skill risks persist across agent backbones and distillation protocols, extending from explicit attributes to communication styles and personality traits.

FinIndices 是一个大规模基准，在未裁剪的财务报表（最长 32K tokens）上评估数据处理保真度，带来显著的零提示增益，验证通过以数据为中心的对齐可部分恢复结构化逻辑。FinIndices is a large-scale benchmark evaluating data-processing fidelity over uncropped financial statements (up to 32K tokens) and yields substantial zero-hint gains, validating that structured logic can be partially restored via data-centric alignment.

提出 ChronoLens，结合冻结的多语言语言模型、特征对齐的 crosscoder 与事后语言学干预，应用于来自五个议会传统、跨越 1803–2026 年的 4498 万篇文档和约 172 亿 tokens，表明历史语言变化是一个结构化的、多维度的过程。ChronoLens is introduced, a framework that combines frozen multilingual language models, feature-aligned crosscoders, and post-hoc linguistic interventions, and applies it to 44.98 million documents and approximately 17.2 billion tokens from five parliamentary traditions spanning 1803--2026, showing that historical language change is a structured, multidimensional process.

提出 GDPevo，一种基于 GDP 相关企业工作流的 evolution-native 基准，并配套完全自动化的数据生成流水线；表现最佳的进化 agent 仍远低于全信息 oracle 上限，表明当前 agent 的自进化能力远未充分实现。GDPevo is presented, an evolution-native benchmark grounded in GDP-related enterprise workflows, together with the fully automated data pipeline that generates it, and the best evolved agents remain far below the fully informed oracle ceiling, indicating that the self-evolution ability of current agents remains far from fully realized.

FinanceHarness 是一个运行金融工具与从业者引导工作流的框架，端到端自动化金融深度研究：环境与数据构建、Agent 执行循环以及奖励建模；FinanceGym 包含论点驱动的研报问题与评分标准，结合 pre-cutoff 与 post-cutoff 准则。FinanceHarness is presented, a harness that runs finance-oriented tools and practitioner-guided workflows, automating financial deep research end to end: environment and data construction, the agent execution loop, and reward modeling, and FinanceGym, comprising thesis-driven research questions and rubrics that combine pre-cutoff and post-cutoff criteria.

MameLoshnLM 是首个专为意第绪语构建的 8B 参数开源语言模型，既为意第绪语 NLP 提供基础，也为历史悠久但数字化程度不足的语言建模开发提供可复用的实践模板。MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish, is presented, providing both a foundation for Yiddish NLP and a practical template for language model development in historically rich but digitally underrepresented languages.

提出 DataSpace，一个基准，用于评估数据 Agent 在任务本地异构工作空间中生成可验证表格结果的能力，并指出提升数据 Agent 可靠性的关键挑战。DataSpace, a benchmark in which data agents produce verifiable tabular results from task-local heterogeneous workspaces, is introduced and key challenges for improving data-agent reliability are identified.

通过分别评估论文检索、证据 grounding 与答案准确性，LitTraceQA 为生成可验证答案、而非无依据摘要的科学 QA 系统提供了测试基准。By evaluating paper retrieval, evidence grounding, and answer accuracy separately, LitTraceQA provides a testbed for scientific QA systems that produce verifiable answers rather than unsupported summaries.

本文介绍了 CodeXGLUE，一个基准数据集，旨在推动面向程序理解与生成的机器学习研究，涵盖 14 个数据集上的 10 项任务，并提供模型评估与比较的平台。This paper introduces CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation that includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison.

于 2018 年 8 月 28 日中午 12:15 在 Pettit 微电子研究中心 102 A/B 室进行报告。Presented on August 28, 2018 at 12:15 p.m. in the Pettit Microelectronics Research Center, Room 102 A/B.

本文提出 MatrAIx，一个面向异构用户的群体规模模拟用户评估基础设施，为使用多样化模拟人类用户评估 AI 系统和数字产品提供端到端支撑。MatrAIx is introduced, a population-scale simulated-user evaluation infrastructure for testing AI systems and digital products with heterogeneous users and provides an end-to-end infrastructure for evaluating AI systems and digital products with diverse simulated human users.

Evo-Bench 是首个跨 Search、Office 和 General Agent 领域评估模型内在 harness 演化能力的基准，并暴露了早期饱和等关键时序异常，同时证明所合成的 harness 是高度可迁移的推理结构，能持续提升多样化策略模型。Evo-Bench is the first benchmark designed to evaluate models'intrinsic harness-evolving capabilities across Search, Office, and General agent domains, and exposes critical temporal anomalies like early saturation, while demonstrating that the synthesized harnesses act as highly transferable reasoning structures, consistently boosting diverse policy models.

本文提出 MMOOC，一个用于评估 MLLMs 拒答与鲁棒回答能力的大规模 benchmark，并引入 LLM-as-a-Judge 指标来衡量模型推理的正确性。This work presents MMOOC, a large-scale benchmark for evaluating refusal and robust answering abilities of MLLMs, and introduces an LLM-as-a-Judge metric to assess the correctness of model reasoning.

本文倡导源对比评估，并构建了 Cultivar——FLORES 的本地化子集，可用于特定 locale 的翻译评估；研究发现：MT 专用模型鲁棒性较差，少数模型可能对 FLORES 存在过拟合，且模型普遍更擅长翻译美国 locale 的内容，而非其他 locale，无论语种如何。This work advocates for source-contrastive evaluation and instantiates Cultivar, a localised subset of FLORES, which enables locale-specific translation evaluation and finds that MT-specialised models are less robust, a few models potentially overfit FLORES, and models tend to translate US content better than that of other locales, regardless of language.

面向策略性优化下的可测性设计指南：保留探针仅在不可枚举轴上保持有效性；门控必须衡量保留集性能，而非仅正确性；迁移率只有在附带每类失败机制评级时才可解读。Design guidance for measurement under strategic optimization is distill design guidance for measurement under strategic optimization: held-out probes retain validity only on non-enumerable axes; gates must measure held-out performance, not just correctness; and a transfer rate is interpretable only with per-failure mechanism grades.

本文实现了对时间序列数据集相似度方法的系统化、可复现比较，提供灵活的扩展性以添加自定义数据集、相似度方法及下游时间序列任务，并通过集成的时间序列数据集归约器对数据集级和序列级相似度方法进行一致评估。This work enables systematic and reproducible comparisons of time-series dataset similarity methods, flexible extensibility for users to add customized datasets, similarity methods, and downstream time-series tasks, and consistent evaluation of both dataset-level and series-level similarity methods through integrated time-series dataset reducers.

本文提出解码层禁忌 (Decoding-Level Taboo)，一种零提示的诊断式压力测试，在运行时直接干预 logit 空间，于词边界处动态遮蔽主要候选 token，强制模型进行迂回表达。Decoding-Level Taboo is introduced, a zero-prompt diagnostic stress test that intervenes directly in logit space at runtime, forcing models out of their nominal paths by dynamically masking primary candidate tokens at word boundaries, forcing machine circumlocution.

360CityArena 为真实感城市区域导航与空间推理提供了必要且具有挑战性的测试平台；基于 SOTA LMM 智能体的评估显示，即使是最强的模型 Gemini 2.5 Flash，其表现仍远低于人类水平。360CityArena provides a necessary and challenging testbed for photorealistic urban-district navigation and spatial reasoning, and evaluation using state-of-the-art LMM-based agents shows that even the strongest model, Gemini 2.5 Flash, performs far below human level.

本文将该挑战形式化为叙事承诺保持 (Narrative Commitment Preservation, NCP)，并提出 NCP-Bench：一个基于电影剧情梗概构建的包含 100 个叙事环境的基准，每个环境均提供可在玩家智能体与叙述者智能体交互过程中自动检查的结构化叙事规范。This work forms this challenge as Narrative Commitment Preservation (NCP), and introduces NCP-Bench, a benchmark of 100 narrative environments derived from movie synopses that each environment includes a structured narrative specification that can automatically check throughout the interaction between the player agent and the narrator agent.

基于 CPI-Bench 对主流图像编辑模型的评测结果显示，CPI-Bench 增强了模型间的性能区分度；排名分析表明 CPI-Bench 与 Arena Image Edit Leaderboard 的对齐度最高，与公开人类偏好排名具有更强的一致性。Evaluation results of mainstream image editing models based on CPI-Bench demonstrate that CPI-Bench enhances performance differentiation among models, and ranking analysis reveals that CPI-Bench achieves the highest alignment with the Arena Image Edit Leaderboard, indicating stronger consistency with public human preference rankings.

在对 1,000 只美股进行小时级收益预测时，我们观察到一种意外现象：预测结果近乎平坦，且截面相关性衡量的股票排序能力很差，我们将其称为"预测坍缩"。令人惊讶的是，在同一设置下对交易量进行预测时，该现象基本消失。我们在多种时序基础模型（TSFMs）、12 个深度学习预测模型以及 97 个公开基准配置中系统考察了这一现象，发现其与目标可预测性密切相关，并识别出背后的两类成因：低 p……When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation. We call this forecast collapse. Surprisingly, the phenomenon largely disappears when forecasting trading volume under the same setting. We investigate forecast collapse across time-series foundation models (TSFMs), twelve deep-learning forecasting models, and 97 public benchmark configurations, and find that it is closely tied to target predictability. We identify two distinct reasons behind it: low p

一个机器人过程评估工具包，将 rollout 视频转化为稠密进度曲线并衍生多项细粒度指标；引入 RoboPulse++ 用于评估过程奖励模型（PRM）的可靠性，为评测者提供更准确的测试平台。A toolkit for robot process assessment that turns rollout videos into dense progress curves and derives multiple fine metrics, and introduces RoboPulse++ to evaluate the reliability of process reward models (PRM), providing evaluators with a more accurate testing platform.

提出 MobileMem，一个面向设备端长期记忆研究的 benchmark 和框架，基于长达一年的移动端经验集合，使 agent 能够记忆过去、理解当下并适应未来。This work introduces MobileMem, a benchmark and framework for studying on-device long-term memory, grounded in a year-scale collection of mobile experiences, and enables agents to remember the past, understand the present, and adapt to the future.

提出 UNMASK，一个全自动 pipeline，可在无需额外人工标注的情况下发现、因果验证并缓解文本分类器中的伪相关，并证明其发现与验证阶段可泛化至奖励模型的偏好数据。U N M ASK is presented, a fully automated pipeline that discovers, causally verifies, and mitigates spurious correlations in text classifiers without additional human annotation, and demonstrates that the discovery and validation stages generalize to reward model preference data.

提出 Apodex Discovery，一个通过 heavy-duty solver 构建和评估发现型 AI 的框架；该 solver 包含一个 foundation model、harness、工具和控制策略，用于执行长期的、有状态的、可验证的探索。This work introduces Apodex Discovery, a framework for building and evaluating discoverative AI through the heavy-duty solver, a system comprising a foundation model, harness, tools, and control policies that pursues extended, stateful, verifiable investigations.

提出 HarnessEval-W，一个 agentified 的评估 pipeline，将 LLM 生态中的 harness 范式引入 world model 基准测试，并在 330 个评估用例上对 18 个代表性 world model 进行了评估。This work introduces HarnessEval-W, an agentified evaluation pipeline that brings the harness paradigm from the LLM ecosystem to world model benchmarking, and applies HarnessEval-W to 18 representative world models over 330 evaluation cases.

语言模型以一种可被报告的形式持有潜在量，并且当任务需要灵活复用该量时，更多该量的信息会以这种形式存在。Language models hold latent quantities in a form they can report on, and more of a quantity is present in that form when the task requires reusing it flexibly when the task requires reusing it flexibly.