论文

本工作提出 12 个具体指标，从一致性、鲁棒性、可预测性和安全性四个关键维度分解 Agent 可靠性，可与传统评估互补，并提供用于分析 Agent 表现、退化与失败方式的工具。This work proposes twelve concrete metrics that decompose agent reliability along four key dimensions: consistency, robustness, predictability, and safety, which complement traditional evaluations while offering tools for reasoning about how agents perform, degrade, and fail.

本文提出一个 Agentic RAG 框架，使 LLM 能够使用逻辑表达式构建检索意图，同时将检索后端简化为基于倒排索引的系统，并表明将检索过程锚定在逻辑查询上可显著降低生成响应中的幻觉。This paper proposes an agentic RAG framework that enables LLMs to formulate retrieval intents using logical expressions while simplifying the retrieval backend to an inverted-index-based system, and shows that anchoring the retrieval process in logical queries substantially reduces hallucinations in generated responses.

本文对一个自 2026 年 3 月起持续运行的个人助理 Agent 运行时中的静默失败进行纵向研究，该系统包含约 40 个定时任务、8 个 LLM 提供商、一个工具治理代理以及一个知识库记忆层，由 4,286 个单元测试和 827 项治理检查守护。A longitudinal study of silent failures in a personal-assistant agent runtime in continuous production since March 2026, with roughly 40 scheduled jobs, 8 LLM providers, a tool-governance proxy, and a knowledge-base memory plane, defended by 4,286 unit tests and 827 governance checks is presented.

结果表明，纯视觉设置会降低准确率并增加 token 成本，因为 Agent 缺乏足够的符号化细节，需通过重复的视觉查询进行补偿；研究指向一种面向下一代编码 Agent 的实用文本与视觉混合设计。The results show that a strictly vision-only setup degrades accuracy and increases token cost, because agents lack sufficient symbolic detail and compensate with repeated visual queries, and point to a practical hybrid text-and-vision design for next-generation coding agents.

认为（该早期版本的）GPT-4 属于新一代具备更通用智能的 LLM（与 ChatGPT、谷歌 PaLM 等并列），并讨论了这些模型不断增强的能力及其影响。It is argued that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models, and the rising capabilities and implications of these models are discussed.

发现对 GPT 语言模型进行重复采样，是为困难 prompt 生成可用解的一种出乎意料的有效策略，并讨论了部署强大代码生成技术在安全性、安全保障与经济等方面的潜在更广泛影响。It is found that repeated sampling from the GPT language model is a surprisingly effective strategy for producing working solutions to difficult prompts, and the potential broader impacts of deploying powerful code generation technologies, covering safety, security, and economics are discussed.

提出 Flamingo，一个 VLM 系列，能够桥接强大的纯视觉与纯语言预训练模型，处理任意交错排列的图文序列，并无缝接收图像或视频作为输入。This work introduces Flamingo, a family of Visual Language Models (VLM) with this ability to bridge powerful pretrained vision-only and language-only models, handle sequences of arbitrarily interleaved visual and textual data, and seamlessly ingest images or videos as inputs.

该综述细致梳理了 RAG 范式的演进，涵盖 Naive RAG、Advanced RAG 与 Modular RAG，并对 RAG 框架的三大基础支柱——检索、生成与增强技术——进行了深入审视。This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG, and meticulously scrutinizes the tripartite foundation of RAG frameworks, which includes the retrieval, the generation and the augmentation techniques.

该综述全面回顾了生成模型的历史与基本组件，以及 AIGC 在单模态交互与多模态交互方向的最新进展，并介绍了文本与图像生成任务及相关模型。This survey provides a comprehensive review on the history of generative models, and basic components, recent advances in AIGC from unimmodal interaction and multimodal interaction, and introduces the generation tasks and relative models of text and image.

提出 BloombergGPT，一个 500 亿参数的语言模型，在广泛的金融数据上训练而成，并基于 Bloomberg 丰富的数据源构建了包含 3630 亿 token 的数据集，可能是迄今最大的领域专用数据集。This work presents BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data, and constructs a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet.

一篇关于基于 LLM 的 Agent 的全面综述，追溯了 Agent 概念从其哲学起源到在 AI 中的发展历程，解释了为何 LLM 适合作为 Agent 的基础，并提出一个包含三个核心组件的通用框架：大脑、感知与行动。A comprehensive survey on LLM-based agents, tracing the concept of agents from its philosophical origins to its development in AI, and explaining why LLMs are suitable foundations for agents, and presenting a general framework, comprising three main components: brain, perception, and action.

综合评估表明，DeepSeek-V3 优于其他开源模型，并达到与领先闭源模型相当的性能。Comprehensive evaluations reveal that DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models.

本文综述了一些最具代表性的 LLM，包括三大主流 LLM 家族（GPT、LLaMA、PaLM），讨论其特性、贡献与局限性，并概述了构建与增强 LLM 的相关技术。This paper reviews some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discusses their characteristics, contributions and limitations, and gives an overview of techniques developed to build, and augment LLMs.

本文对 LLM 的评估方法进行了全面综述，围绕三个关键维度展开：评估什么、在何处评估、如何评估，并为 LLM 评估领域的研究者提供了宝贵洞见。This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate, where to evaluate, and how to evaluate, and offers invaluable insights to researchers in the realm of LLMs evaluation.

分析表明，借助 LLM，美国约 15% 的工作任务可在保持同等质量的前提下显著提速完成，意味着 LLM 驱动的软件将对底层模型经济影响的规模化产生实质性作用。The analysis suggests that, with access to an LLM, about 15% of all worker tasks in the US could be completed significantly faster at the same level of quality, implying that LLM-powered software will have a substantial effect on scaling the economic impacts of the underlying models.

研究发现，在上述多个维度上进行的指令微调可显著提升多种模型类别（PaLM、T5、U-PaLM）、多种提示设定以及多种评测基准（MMLU、BBH、TyDiQA、MGSM、开放式生成）上的表现。It is found that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups, and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).

本文描述了一份以模式形式呈现的 prompt 工程技巧目录，这些技巧已被用于解决与 LLMs 对话时的常见问题，以改进 LLM 对话的输出。A catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs to improve the outputs of LLM conversations is described.

研究发现，ChatGPT 最成功地被用作数学助手，用于查询事实、充当数学搜索引擎和知识库接口；GPT-4 还可被用于本科水平的数学问题，但在研究生难度的题目上表现不佳。It is found that ChatGPT can be used most successfully as a mathematical assistant for querying facts, acting as a Mathematical search engine and knowledge base interface, and GPT-4 can additionally be used for undergraduate-level mathematics but fails on graduate-level difficulty.

本文进行了迄今为止对 Code LLMs 最全面的评估，结果显示 StarCoderBase 在支持多编程语言的开放 Code LLMs 中表现最优，并且能够匹敌或超越 OpenAI code-cushman-001 模型。This work performs the most comprehensive evaluation of Code LLMs to date and shows that StarCoderBase outperforms every open Code LLM that supports multiple programming languages and matches or outperforms the OpenAI code-cushman-001 model.

本文显著推进了针对已对齐语言模型的对抗攻击 SOTA，并提出了关于如何防止此类系统生成不良信息的重要问题。This work significantly advances the state-of-the-art in adversarial attacks against aligned language models, raising important questions about how such systems can be prevented from producing objectionable information.

本文旨在建立一套系统化的开放问题与应用成果清单，以便机器学习研究者更快地理解该领域的现状并开展有效工作。This paper aims to establish a systematic set of open problems and application successes so that ML researchers can comprehend the field's current state more quickly and become productive.

EvalPlus——一个用于严格基准测试 LLM 生成代码功能正确性的代码合成评估框架，通过 LLM 与基于 mutation 的策略驱动的自动测试输入生成器，为给定评估数据集补充大量新生成的测试用例。EvalPlus -- a code synthesis evaluation framework to rigorously benchmark the functional correctness of LLM-synthesized code and augments a given evaluation dataset with large amounts of test-cases newly produced by an automatic test input generator, powered by both LLM and mutation-based strategies.

实证研究表明 AutoGen 框架在多个示例应用中有效，应用领域涵盖数学、编码、问答、运筹学、在线决策、娱乐等。Empirical studies demonstrate the effectiveness of the AutoGen framework in many example applications, with domains ranging from mathematics, coding, question answering, operations research, online decision-making, entertainment, etc.

PaLM 2 是一个新的 SOTA 语言模型，相比其前身 PaLM 具有更强的多语言和推理能力，并具备更高的计算效率，能够在不增加额外开销或影响其他能力的前提下在推理时控制输出毒性。PaLM 2 is a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM and enables inference-time control over toxicity without additional overhead or impact on other capabilities.

名为 PaLM 的 540 亿参数、密集激活的 Transformer 语言模型取得了突破性性能，在一系列多步推理任务上超越了微调后的 SOTA，并在最近发布的 BIG-bench 基准上超越了人类平均水平。A 540-billion parameter, densely activated, Transformer language model, which is called PaLM achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark.

采用迭代的在线训练模式，按周节奏用新的人类反馈数据更新偏好模型与 RL 策略，并发现 RL 奖励与策略相对其初始化的 KL 散度平方根之间近似呈线性关系。An iterated online mode of training, where preference models and RL policies are updated on a weekly cadence with fresh human feedback data, and a roughly linear relation between the RL reward and the square root of the KL divergence between the policy and its initialization is identified.

研究表明，APE 生成的提示词既可引导模型趋向真实性和/或信息量，也可通过将其前置拼接到标准上下文学习提示词之前来提升少样本学习性能。It is shown that APE-engineered prompts can be applied to steer models toward truthfulness and/or informativeness, as well as to improve few-shot learning performance by simply prepending them to standard in-context learning prompts.

本文提出 Atlas，一个经过精心设计并预训练的检索增强大语言模型，能以极少训练样例学习知识密集型任务，并研究了文档索引内容的影响，表明该索引可便捷地更新。This work presents Atlas, a carefully designed and pre-trained retrieval augmented language model able to learn knowledge intensive tasks with very few training examples, and studies the impact of the content of the document index, showing that it can easily be updated.