Papers · organized/paper_cards

论文

32 张论文卡片 · 评测基准 · 方法

开放获取 全部 绿色 · 677
1. Recursive Agent Harnesses (RAH)
1. Recursive Agent Harnesses(RAH)
arXiv:2606.13643 评测基准 方法 OA · 绿色 被引 2 · S2

本文命名并研究这两条研究脉络之间的模式:其递归单元是配备文件系统工具、代码执行与规划的完整 Agent harness,而非无工具的模型调用,并给出针对长上下文推理的受控评估。This work names and studies the pattern between these two lines of work, where the recursive unit is a full agent harness with filesystem tools, code execution, and planning rather than a model call with no tools, and provides a controlled evaluation on long-context reasoning.

🟡 保留 4:"The Last Harness" — Meta-Evolution 双层循环
arXiv:2604.21003 评测基准 方法 OA · 绿色 被引 2 · S2

一个两级框架将手动 harness 工程转变为自动化 harness 工程,并更进一步——将"自动化本身的设计"也自动化。A two-level framework shifts manual harness engineering into automated harness engineering, and takes one step further --automating the design of the automation itself.

🔴 保留 3:Agentic Harness Engineering (AHE) — arXiv 实证论文
🔴 保留 3:Agentic Harness Engineering(AHE)— arXiv 实证论文
arXiv:2604.25850 评测基准 方法 OA · 绿色 被引 56 · S2

提出 Agentic Harness Engineering(AHE),一个通过三个相互匹配的 observability 支柱应对 harness 工程挑战的闭环,将每一次编辑转化为可证伪的契约,使 harness 演进能够自主进行而不退化为试错。Agentic Harness Engineering (AHE) is introduced, a closed loop that addresses harness engineering challenges through three matched observability pillars that turn every edit into a falsifiable contract, so harness evolution proceeds autonomously without collapsing into trial-and-error.

Agent runtime / security / harness 补充候选
Agent runtime / security / harness 补充候选
arXiv:2603.25723 评测基准 方法 OA · 绿色 被引 34 · S2

本文提出 Natural-Language Agent Harnesses,即可编辑的、描述运行级 harness 策略的文档,以及 Intelligent Harness Runtime(IHR),一个将上述文档解释为 agent 调用、交接、状态更新、验证门控与 artifact 契约的共享运行时。This paper introduces Natural-Language Agent Harnesses, editable documents that describe run-level harness policy, and Intelligent Harness Runtime (IHR), a shared runtime that interprets these documents into agent calls, handoffs, state updates, validation gates, and artifact contracts.

Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
大语言模型是否在玩"六度分隔"?长上下文流形中的拓扑压缩度量
arXiv:2608.17950 评测基准 方法 被引 0 · S2

该工作数学形式化了 Transformer 如何执行抽象推理,并提出一种新颖的严格几何签名用于评估事实可靠性,证明了深度 LLM 潜空间天然组织为小世界网络。This work mathematically formalizes how transformers execute abstract reasoning and provides a novel, strictly geometric signature for evaluating factual reliability, proving that deep LLM latent spaces natively organize into Small-World networks.

Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence
Zetta ζ:面向自进化物理智能的高效闭环具身 Harness
arXiv:2608.16590 评测基准 方法 被引 0 · S2

本文提出 Zetta,一种闭环具身 harness,在保持基础策略冻结的同时在线演化基于代码的运行时评判器与恢复技能,表明闭环 harness 的自进化为可靠的物理智能开辟了一条可扩展的路径Zetta is presented, a closed-loop embodied harness that evolves code-based runtime critics and recovery skills online while keeping the base policy frozen, and shows that closed-loop harness self-evolution opens a scaling path for reliable physical intelligence.

TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicity
TinyCast:基于计算周期性的概率性零样本预测
arXiv:2608.15767 评测基准 方法

我们提出 TinyCast,一个注意力无关的零样本预测器,仅用 146,505 个参数输出预测分布,其前提是在该规模下,上下文中的周期结构值得通过计算而非学习方式得到。一个零参数谱检测器给出主导周期,上下文按其相位进行折叠,再由一个膨胀卷积编码器和一个分块自回归分位数解码器建模其余部分。它在 GIFT-Eval 榜单上所有可确认参数量的零样本条目中体积最小;在概率准确性方面,它划定了 size-accuracy 前沿。We introduce TinyCast, an attention-free zero-shot forecaster that emits a predictive distribution from 146,505 parameters, on the premise that at this size the periodic structure of a context is worth computing rather than learning. A zero-parameter spectral detector supplies the dominant periods, the context is folded on their phase, and a dilated convolutional encoder and a block-autoregressive quantile decoder model the rest. It is smaller than every zero-shot entry on the GIFT-Eval board whose parameter count can be established. On probabilistic accuracy it defines the size-accuracy front

Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection
Task-CoEvolve:通过自适应验证任务选择实现 Harness 高效优化
arXiv:2608.20169 评测基准 方法

我们提出一种通过自适应验证任务选择来实现 LLM harness 高效优化的新方法。Harness 优化基于验证性能迭代改写 harness 代码,无需更新底层模型权重即可获得显著性能提升。然而现有方法在每次迭代中对固定的验证集进行完整评估,即便某些任务随 harness 演进区分度下降,仍产生高昂的评测成本。我们提出 Task-CoEvolve,通过应对两项挑战使验证任务与 harness 协同演进:seWe present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose Task-CoEvolve, which co-evolves the validation tasks with the harness by addressing two challenges: se

LongWoF-Bench: Evaluating EvoMap Genes for Verifiable Long-Workflow Tasks
LongWoF-Bench:评估 EvoMap Gene 的可验证长工作流任务基准
arXiv:2608.23200 评测基准 方法

大语言模型日益被期望执行复杂工作流,其成功依赖于维持相互依赖的约束并产出满足严格端到端验证的产物。然而成功的执行经验通常在单次运行后就丢失了,迫使后续模型从头重新发现策略和失败模式。我们研究能否通过 EvoMap 将此类经验外部化并复用,其中验证器确认的执行轨迹被整合为结构化的 Gene。为评估该设定,我们引入长工作流基准Large language models are increasingly expected to execute complex workflows whose success depends on maintaining interdependent constraints and producing artifacts that satisfy strict end-to-end verification. Yet successful execution experience is typically lost after a single run, forcing subsequent models to rediscover strategies and failure modes from scratch. We study whether such experience can instead be externalized and reused through EvoMap, where verifier-confirmed execution trajectories are consolidated into structured Gene. To evaluate this setting, we introduce the Long-Workflow B

1️⃣ arXiv · Learning Rate Matters: Vanilla LoRA May Suffice(⭐⭐⭐⭐⭐ 必读)
学习率至关重要:Vanilla LoRA 可能已足够
arXiv:2602.04998 评测基准 方法 Open MIND OA · 绿色 被引 8 · S2

本文通过大规模超参数搜索,系统地重新评估了 Vanilla LoRA 以及九个代表性 LoRA 变体,发现不同 LoRA 方法偏好的学习率区间各异,并将最优学习率区间的差异归因于最大 Hessian 特征值的变化,与经典学习理论相吻合。This work systematically re-evaluate nine representative LoRA variants alongside vanilla LoRA through extensive hyperparameter searches, finding that different LoRA methods favor distinct learning rate ranges and attributes the differing optimal learning rate ranges to variations in the largest Hessian eigenvalue, aligning with classical learning theories.

Measuring the Gap Between Human and LLM Research Ideas
衡量人类与 LLM 研究思路之间的差距
arXiv:2607.01233 评测基准 方法 OA · 绿色 被引 3 · S2

结果表明,强大的 LLM 能够产出一系列合理思路,但其范围仍比人类研究品味更窄,并存在系统性偏移。It is suggested that strong LLMs can produce a range of reasonable ideas, but that range remains narrower than, and systematically shifted relative to, human research taste.

Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms
深度强化学习评估与设计范式的原理性分析
arXiv:2607.07769 评测基准 方法 OA · 绿色 被引 0 · S2 + OpenAlex

介绍其潜在原因的理论基础,阐明强化学习算法的渐近性能在性能排名与数据规模之间不存在单调关系。The theoretical foundations of the underlying causes outlining that the asymptotic performance of reinforcement learning algorithms does not have a monotone relationship between performance rankings and data-regimes are introduced.

Length Penalties Make Chain-of-Thought Less Monitorable
长度惩罚使 Chain-of-Thought 更难被监控
arXiv:2607.09786 评测基准 方法 OA · 绿色 被引 0 · S2 + OpenAlex

压缩可减少推理 token 并保持大部分选择题准确率,同时提示影响接近基线——在一个前沿条件下,降低推理成本移除的证据比单纯缩短轨迹所预期的更多。Compression reduces reasoning tokens and preserves most multiple choice accuracy, while hint influence remains near baseline, in a frontier where reducing reasoning costs removes more evidence than shorter traces alone would predict.

UniVR: Thinking in Visual Space for Unified Visual Reasoning
UniVR:在视觉空间中思考以实现统一视觉推理
arXiv:2607.12800 评测基准 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 UniVR,这是首个从纯视觉演示中同时学习复杂推理、细粒度物理动力学和长期规划的研究,并配套首个在纯视觉协议下评估这些异质能力的综合评测套件。UniVR, the first investigation into simultaneously learning complex reasoning, fine-grained physical dynamics, and long-term planning from pure visual demonstrations, is introduced, the first comprehensive suite to assess these heterogeneous capabilities under a purely visual protocol.

Evaluating Large Language Models Trained on Code
Evaluating Large Language Models Trained on Code
arXiv:2107.03374 评测基准 方法 OA · 绿色 被引 11151 · S2

发现对 GPT 语言模型进行重复采样,是为困难 prompt 生成可用解的一种出乎意料的有效策略,并讨论了部署强大代码生成技术在安全性、安全保障与经济等方面的潜在更广泛影响。It is found that repeated sampling from the GPT language model is a surprisingly effective strategy for producing working solutions to difficult prompts, and the potential broader impacts of deploying powerful code generation technologies, covering safety, security, and economics are discussed.

AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System
AILQA:面向印度法律体系的 AI 驱动法律问答系统评测
arXiv:2607.18825 评测基准 方法 OA · 绿色 被引 1 · S2

在该研究的评估协议下,部分 AI 生成回答获得了高于可用参考答案的评分,尤其当其包含准确且相关的支撑细节时尤为明显。Under the study's evaluation protocol, some AI-generated responses received higher ratings than the available reference answers, particularly when they contained accurate and relevant supporting details, particularly when they contained accurate and relevant supporting details.

Transcription Policy as a Latent Variable: Activating Controllable Verbatim ASR with Word-Level Timing
转写策略即潜变量:通过词级时序激活可控逐字 ASR
arXiv:2607.18934 评测基准 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 Verbatimize 这一新任务,能够以高质量规范化逐字转录对语音语料库进行可扩展的创建与扩充,并通过有监督的 cross-attention 微调,将不流畅语音上的词级时间戳表现提升至超过强制对齐 baseline。Verbatimize is proposed, a new task enabling scalable creation and enrichment of speech corpora with high-quality canonical verbatim transcriptions and supervised cross-attention fine-tuning that improves word-level timestamps on disfluent speech beyond forced-alignment baselines.

Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering
FinMMEval 2026 任务 2 概述:多语言金融短答问答
arXiv:2607.19867 评测基准 方法 OA · 绿色 被引 1 · S2

FinMMEval 2026 Task 2 围绕多语言证据评估金融领域的短答问答,采用文档 RAG、跨语言证据处理、结构化提示、答案压缩与验证策略。FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence over multilingual evidence using document retrieval-augmented generation, cross-lingual evidence handling, structured prompting, answer compression, and validation strategies.

OpenForgeRL: Train Harness-native Agents in Any Environment
OpenForgeRL:在任意环境中训练 Harness 原生 Agent
arXiv:2607.21557 评测基准 方法 被引 2 · S2

提出 OpenForgeRL,一个用于在多样化环境中端到端训练基于 harness 的 Agent 的开源框架,并在多种复杂的 harness 和环境中得到验证,涵盖工具/爪型 Agent 以及多模态 GUI 浏览器和计算机使用 Agent。OpenForgeRL is presented, an open-source framework for training harness-based agents end-to-end in diverse environments, and validated across diverse, complex harnesses and environments, spanning tool/claw-based agents and multimodal GUI browser- and computer-use agents.

Codifying the Judge: Scalable Evaluation via Program Distillation
Codifying the Judge:通过程序蒸馏实现可扩展评估
arXiv:2607.22561 评测基准 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 PAJAMA,该系统将程序合成为 judge,将其决策聚合为联合裁决(joint verdict),并通过 fallback 机制选择性地将低置信度用例升级交由 LLM 处理。PAJAMA is introduced, a system that synthesizes programs as judges, aggregates their decisions into a joint verdict, and incorporates a fallback mechanism to selectively escalate low-confidence cases to an LLM.

DecoEvo: Score-Decoupled Co-Evolution of Solver and Rubric-Generator Skills in Text Space
DecoEvo: 文本空间中 Solver 与 Rubric 生成器技能的分数解耦协同演化
arXiv:2607.25675 评测基准 方法 OA · 绿色 被引 2 · S2

介绍 DecoEvo(Decoupled Co-Evolution),在解耦目标下协同进化一个求解器 skill 和一个 rubric 生成器 skill,优化过程中不使用 gold rubric。DecoEvo (Decoupled Co-Evolution), which co-evolves a solver skill and a rubric-generator skill under decoupled objectives without using gold rubrics during optimization, is introduced.

CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization
CoRT: 用于 token 级 rubric 引导策略优化的反事实回放
arXiv:2607.25659 评测基准 方法 OA · 绿色 被引 0 · S2 + OpenAlex

本文提出 CoRT,一种用于 rubric 条件化 GRPO 的 token 级 credit 加权方法,通过反事实回放在原始 rubric 条件化 prompt 和匹配的免准则 prompt 下对同一样本响应重新打分,并表明策略内部反事实似然对比为响应内 credit 分配提供了有效的训练信号。CoRT is proposed, a token-level credit weighting method for rubric-conditioned GRPO that uses counterfactual replay to rescore the same sampled response under the original rubric-conditioned prompt and a matched criteria-free prompt, and suggests that policy-internal counterfactual likelihood contrasts provide an effective training signal for within-response credit allocation.

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Fairness Pruning:通过差异激活定位 GLU-MLP 层中的人口统计偏差
arXiv:2607.28319 评测基准 方法 被引 0 · S2

对 Fairness Pruning 的实证评估表明,群体偏置处理与模型能力运行在可分离的电路上,奠定了从盲目零化向定向行为调制过渡的方法论基础。Empirical evaluation of Fairness Pruning empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.

Evaluation-Verification Reward for Consistent Multi-Reference Image Editing
用于一致多参考图像编辑的评估-验证奖励
arXiv:2607.29025 评测基准 方法 被引 0 · S2

多维评估-验证奖励(EVR)将评估分解为独立视觉准则;针对每个准则,MLLM Evaluator生成多个候选假设,Verifier在具体视觉证据中grounding每个claim以接受或拒绝,产生可靠且细粒度的奖励信号。A Multi-dimensional Evaluation-Verification Reward (EVR) decomposes evaluation into distinct visual criteria; for each criterion, an MLLM Evaluator generates multiple candidate hypotheses, and a Verifier grounds each claim in concrete visual evidence to accept or reject it, producing reliable and fine-grained reward signals.

RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
RecHarness:用于自演化推荐系统的 Bandit 路由 Agentic Harness
arXiv:2607.29241 评测基准 方法 被引 0 · S2

RecHarness 将优化过程分为两步:bandit 路由器根据历史验证反馈选择下一步修改方向,LLM 在选定方向内生成具体优化假设与可执行代码编辑RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction.

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
标题 -> 标题中文:增是机器,删是人工:LLM 代码编辑中删除回避的度量与缓解
arXiv:2607.28887 评测基准 方法 被引 0 · S2

在后训练中教授删除操作可减少删除回避行为并提升更广泛的代码编辑性能,表明该行为是训练不足而非不可达成Teaching deletion during post-training reduces deletion avoidance and improves broader code-editing performance, suggesting the behavior is undertrained rather than beyond reach.

OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
OSReward:建立跨平台计算机使用 Reward Model 的标准化评估
arXiv:2607.28609 评测基准 方法 被引 0 · S2

提出 OSReward,一个面向真实场景的高质量基准,用于评估 VLM 评判器在 CUA 轨迹上的表现;同时发布一个面向 CUA 社区的、带推理标注的开放轨迹判断语料库,以弥补大规模可靠 CUA 奖励的缺口。OSReward is introduced, a realistic, high-quality benchmark that evaluates VLM judges on CUA trajectories, and an open corpus of reasoning-annotated trajectory judgments for the CUA community, to close the gap in reliable CUA reward at scale.

BERTScore: Evaluating Text Generation with BERT
BERTScore:使用 BERT 评估文本生成
arXiv:1904.09675 评测基准 方法 OA · 绿色 被引 9409 · S2

本工作提出 BERTScore——一种文本生成自动评估指标,与人类判断的相关性更强,并在模型选择性能上优于现有指标。This work proposes BERTScore, an automatic evaluation metric for text generation that correlates better with human judgments and provides stronger model selection performance than existing metrics.

Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
MLLM 能解读"创意跳跃"吗?面向跨概念理解的 C4 基准
arXiv:2608.06501 评测基准 方法 被引 0 · S2

提出 C4,一个受认知启发的成语跨概念创造力评估框架,揭示了当前 MLLM 在通过跨概念关系解码创造性编码语义方面存在的显著差距。C4, a cognition-inspired evaluation framework for Chengyu (Chinese idiom)-based Cross-Concept Creativity, is introduced, exposing a substantial gap in how current MLLMs decode creatively encoded meaning through cross-concept relations.

Complementary Matrix-Gated QKAN Fast-Weight Programmers for Quantum Dynamics Forecasting
用于量子动力学预测的互补矩阵门控 QKAN 快权重编程器
arXiv:2607.27945 评测基准 方法 被引 0 · S2

本文提出 Self-Modulating QKAN-based FWPs,用低秩生成的逐元素调制(作用于新提案分支、有界旧状态分支或两者)替代该广播门,并提出 Complementary Matrix Gating (CMG),在保持标量门控的有界凸更新和仿射前缀扫描结构的同时提供逐坐标的记忆控制。This work introduces Self-Modulating QKAN-based FWPs, which replace this broadcast gate with low-rank-generated element-wise modulation of the new-proposal branch, a bounded old-state branch, or both, and proposes Complementary Matrix Gating (CMG), which provides coordinate-wise memory control while preserving the bounded convex update and affine prefix-scan structure of scalar gating.

BDH-CQ: In-Context Learning with Recurrent Latent Reasoning
BDH-CQ:基于循环潜变量推理的上下文学习
arXiv:2608.09888 评测基准 方法 被引 0 · S2

本文提出 BDH-CQ,一个结合上下文学习与循环潜在推理的推理模型,通过在高维潜在空间中的迭代计算来求解查询,且无需将中间推理外化为语言。BDH-CQ is introduced, a reasoning model that combines in-context learning with recurrent latent reasoning that solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning.

Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search
Large Discovery Models:基于经验、依托模型驱动的开放式搜索
arXiv:2608.15669 评测基准 方法 被引 0 · S2

提出 Large Discovery Model (LDM),一种经验驱动的循环架构,将生成模型与贝叶斯非参数奖励代理模型耦合,产生一种感知不确定性的价值,用于引导候选的生成、精炼与选择。This work introduces the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection.