论文

TritonForge 是一个面向自动化 Triton kernel 优化的 profiling 引导框架，融合 kernel 分析、运行时 profiling 与迭代式代码转换以简化优化流程，并为自动化 GPU 性能优化领域的未来研究奠定基础。TritonForge, a profiling-guided framework for automated Triton kernel optimization that integrates kernel analysis, runtime profiling, and iterative code transformation to streamline the optimization process and provides a foundation for future research in automated GPU performance optimization.

本文探讨容器化、微服务、动态调度等 Cloud Native 技术如何从根本上提升 LLM 推理服务，并展示 Cloud Native 系统在高需求场景下实现更高效资源分配、降低延迟与提升吞吐的能力。This article explores how Cloud Native technologies, such as containerization, microservices, and dynamic scheduling, can fundamentally improve LLM inference serving and demonstrates how a Cloud Native system enables more efficient resource allocation, reduces latency, and enhances throughput in high-demand scenarios.

本文首次对 vLLM 启动延迟进行了详细的性能表征，并构建了一个轻量级分析模型，能够针对给定硬件配置准确预测 vLLM 的启动延迟，为大规模推理环境中的资源规划提供了可操作的指导。This paper presents the first detailed performance characterization of vLLM startup latency and develops a lightweight analytical model that accurately predicts vLLM's startup latency for a given hardware configuration, providing actionable guidance for resource planning in large-scale inference environments.

NetKV，一种使用该 oracle 信息的 O(|D|) 每请求贪心策略，其层级排序被证明对过时遥测数据具有鲁棒性；并证明随着上下文长度增长，忽略网络项会使仅缓存感知的调度任意次优。NetKV, the O(|D|) per-request greedy that consumes this oracle, has tier rankings that are provably robust to stale telemetry, and it is proved that ignoring the network term renders cache-aware-only scheduling arbitrarily suboptimal as context length grows.

本工作提出 LMCACHE，首个也是目前最高效的开源 KV 缓存方案，可将现代 LLM 引擎生成的 KV 缓存从 GPU 显存中提取并存储，并跨引擎和查询共享。This work presents LMCACHE, the first and so far the most efficient open-source KV caching solution, which extracts and stores KV caches generated by modern LLM engines out of the GPU memory and shares them across engines and queries.

提出 KVpop，通过直接监督保留或丢弃决策来学习固定预算的 KV 淘汰策略，并引入基于延迟记忆的打分器，在已学习的淘汰方法中独有地延迟固定步数的打分，以利用近期上下文。KVpop is introduced, which learns a fixed-budget KV eviction policy by directly supervising the keep-or-drop decision, and introduces a delayed memory-based scorer that, uniquely among learned eviction methods, defers scoring for a fixed number of steps to exploit near-future context.

提出流形约束假设（Manifold Constraint Hypothesis, MCH）：若自然表征集中于一个结构化的低维流形上，则干预应被约束在该流形内，从而在干预过程中更好地保留表征中编码的其他信息。The Manifold Constraint Hypothesis (MCH) is proposed: if natural representations concentrate on a structured, lower-dimensional manifold, then interventions should be constrained to that manifold and better preserve other information encoded in the representation during interventions.

提出 Simplified Sparse Attention——一种更简洁的稀疏注意力方案，无需任何架构改动，在检索增强生成中优于全注意力；并扩展为分层 gist-of-gist 变体，在高达 32× 的高压缩比下保持或提升精度，同时实现对数级解码复杂度。Simplified Sparse Attention is introduced, a simpler approach to sparse attention that requires no architectural changes that outperforms full attention in retrieval-augmented generation and extends to a hierarchical gist-of-gist variant that achieves log-linear decoding complexity while maintaining or improving accuracy at high compression ratios up to 32x.

提出 Erase-then-Delta Attention (EDA)，一种将"在哪里擦除"与"在哪里写入"解耦的内存更新规则；研究表明循环记忆模型不仅应决定写入什么，还应决定擦除哪些陈旧信息以及擦除的位置。Erase-then-Delta Attention (EDA), a memory update rule that decouples where to erase from where to write, is proposed, suggesting that recurrent memory models should decide not only what to write, but also what stale information to erase and where.

本文从前瞻视角重新审视 token 重要性，提出衡量压缩 token 对未来上下文影响的新指标 Forward Influence，以及融合信息论信号的熵感知 KV cache 压缩框架 InfoKV。This paper revisits token importance from a forward-looking perspective and introduces Forward Influence, a metric that measures how compressed tokens affect future contexts, and proposes InfoKV, an entropy-aware KV cache compression framework that incorporates information-theoretic signals.

提出 SAC，这是首个针对稀疏注意力模型优化的高效解耦 KV Cache 系统，利用 CXL（Compute Express Link）的低延迟、cache-line 粒度 load/store 语义，将基于 CXL 的解耦确立为新兴稀疏注意力模型的优越基础设施。SAC is proposed, the first efficient disaggregated KV cache system optimized for sparse attention models, which leverages the low-latency, cache-line granularity load/store semantics of Compute Express Link (CXL), establishing CXL-based disaggregation as the superior infrastructure for emerging sparse attention models.

本文通过收益–成本视角研究 PLT 循环次数选择：额外循环可精炼表示，但 CLP 也会在每次循环边界引入位置错配，由此解释 PLT 在两次循环时趋于饱和的现象，并为循环次数选择提供诊断依据。This study studies PLT loop-count selection through a gain--cost view: an extra loop may refine representations, but CLP also introduces a positional mismatch at each loop boundary, explaining PLT's saturation at two loops and providing diagnostics for loop-count selection.

提出 Sparse Delta Memory，一种通过稀疏寻址方案将门控线性 RNN 隐状态容量扩展数个数量级的架构，在上下文学习与长上下文检索任务上显著提升性能。Sparse Delta Memory is introduced, an architecture that scales the hidden state of gated linear RNNs to orders of magnitude higher capacity using a sparse addressing scheme and significantly improves performance on in-context learning and long-context retrieval tasks.

本文对 softmax 注意力与四种近期的循环线性注意力架构（DeltaNet、Gated DeltaNet、Kimi Delta Attention 与 Gated DeltaNet-2）进行对比研究，明确阐述它们在表达能力、记忆衰减、擦写控制、训练吞吐量与实现复杂度上的差异。A comparative study of softmax attention and four recent recurrent linear-attention architectures: DeltaNet, Gated DeltaNet, Kimi Delta Attention, and Gated DeltaNet-2 is presented, making explicit how they differ in expressivity, memory decay, erase and write control, training throughput, and implementation complexity.

Soofi S 30B-A3B 是一个主权开源的 MoE 混合 Mamba Transformer 德英基础模型，在作者的对比中超越了所有欧洲主权基线，包括活跃参数量远超自身的模型。Soofi S 30B-A3B is a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English that outperforms every European sovereign baseline in the authors' comparison, including ones far larger in active parameters.

GigaWorld-Policy-0.5 在保留未来视觉动力学训练收益的同时提升了机器人控制的推理效率，并引入 Mixture-of-Transformers 架构，将视觉动力学建模与动作生成分离到专门的专家模块中。GigaWorld-Policy-0.5 preserves the training benefits of future visual dynamics while improving inference efficiency for robot control, and introduces a Mixture-of-Transformers architecture that separates visual dynamics modeling and action generation into specialized experts.

研究表明模型具备相关子群体知识，但难以稳定传递到聚合估计中，这一差距使统计自一致性成为评估 LLM 的尚未饱和、无需参考的准则。It is suggested that models possess relevant subpopulation knowledge but do not reliably propagate it into aggregate estimates, and this gap establishes statistical self-consistency as an unsaturated, reference-free criterion for evaluating LLMs.

本文旨在为研究者与从业者提供一份快速、全面的参考，通过对现有工作的广泛、信息密集型总结来汲取洞见，以推动 LLM 研究的发展。This review article is intended to provide a quick, comprehensive reference for the researchers and practitioners to draw insights from extensive, informative summaries of the existing works to advance the LLM research.

提出 BloombergGPT，一个 500 亿参数的语言模型，在广泛的金融数据上训练而成，并基于 Bloomberg 丰富的数据源构建了包含 3630 亿 token 的数据集，可能是迄今最大的领域专用数据集。This work presents BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data, and constructs a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet.

本文试图以简洁的方式描述受生成式 AI 影响的主要行业与模型，并给出近期主要生成式模型的分类体系。This work consists on an attempt to describe in a concise way the main models are sectors that are affected by generative AI and to provide a taxonomy of the main generative models published recently.

综合评估表明，DeepSeek-V3 优于其他开源模型，并达到与领先闭源模型相当的性能。Comprehensive evaluations reveal that DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models.

本文综述了一些最具代表性的 LLM，包括三大主流 LLM 家族（GPT、LLaMA、PaLM），讨论其特性、贡献与局限性，并概述了构建与增强 LLM 的相关技术。This paper reviews some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discusses their characteristics, contributions and limitations, and gives an overview of techniques developed to build, and augment LLMs.

这些发现表明，即便长上下文评估正从简单检索转向复杂推理，对相关证据的准确 grounding 仍是一项不可或缺且仍有大幅提升空间的能力。These findings suggest that, even as long-context evaluation shifts from simple retrieval toward complex reasoning, accurate grounding in relevant evidence remains an indispensable capability with substantial room for improvement.

本文描述了 TensorFlow 接口及 Google 构建的该接口实现，已被用于开展研究，并在计算机科学及其他十余个领域中将机器学习系统部署至生产环境。The TensorFlow interface and an implementation of that interface that is built at Google are described, which has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields.

本文推出参数规模从 7B 到 65B 的基础语言模型集合 LLaMA，并证明完全使用公开数据集即可训练出 SOTA 模型，无需依赖专有或不可获取的数据。LLaMA, a collection of foundation language models ranging from 7B to 65B parameters, is introduced and it is shown that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets.

通过从概率分布的动态 nucleus 中采样文本，可在有效截断不可靠分布尾部的同时保持多样性，使生成文本更接近人类文本质量，在不牺牲流畅性与连贯性的前提下提升多样性。By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.

本文指出基于 Transformer 的次二次时间模型的关键缺陷在于无法执行基于内容的推理，并将选择性 SSM 集成到不包含注意力乃至 MLP 块的简化端到端神经网络架构（Mamba）中。This work identifies that a key weakness of subquadratic-time models based on Transformer architecture is their inability to perform content-based reasoning, and integrates selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba).

该设计将 hypernetwork 的注入能力与目标模型的通用能力解耦，首次实现了对 hypernetwork 架构 scaling law 的严格研究，并提供了首个基于实证的 scaling law，用以指导大语言模型中面向事实推理的 hypernetwork 设计。The design decouples the hypernetwork's injection capacity from the target model's general capability, enabling, for the first time, a rigorous study of scaling laws for hypernetwork architectures, and provides the first empirically grounded scaling laws to guide hypernetworks for factual reasoning in large language models.

GPT-3 在多个 NLP 数据集上取得了强劲表现，包括翻译、问答和完形填空任务，以及若干需要即时推理或领域适应的任务，例如乱序词重组、在句子中使用新词、或执行三位数算术运算。GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic.

本文提出了一种新的解码策略——self-consistency，用于替代思维链 prompt 中使用的朴素贪心解码：首先采样一组多样化的推理路径，而非仅取贪心路径，然后通过对采样路径进行边缘化来选择最一致的答案。This paper proposes a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting that first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths.

本文进行了迄今为止对 Code LLMs 最全面的评估，结果显示 StarCoderBase 在支持多编程语言的开放 Code LLMs 中表现最优，并且能够匹敌或超越 OpenAI code-cushman-001 模型。This work performs the most comprehensive evaluation of Code LLMs to date and shows that StarCoderBase outperforms every open Code LLM that supports multiple programming languages and matches or outperforms the OpenAI code-cushman-001 model.

本文旨在建立一套系统化的开放问题与应用成果清单，以便机器学习研究者更快地理解该领域的现状并开展有效工作。This paper aims to establish a systematic set of open problems and application successes so that ML researchers can comprehend the field's current state more quickly and become productive.

PaLM 2 是一个新的 SOTA 语言模型，相比其前身 PaLM 具有更强的多语言和推理能力，并具备更高的计算效率，能够在不增加额外开销或影响其他能力的前提下在推理时控制输出毒性。PaLM 2 is a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM and enables inference-time control over toxicity without additional overhead or impact on other capabilities.

本文在使用 SOTA CNN 的情况下，实现了 16–24 倍的网络压缩，仅带来 1% 的分类准确率损失，并发现针对存储开销最大的全连接层进行压缩时，向量量化方法相比现有矩阵分解方法具有明显优势。This paper is able to achieve 16-24 times compression of the network with only 1% loss of classification accuracy using the state-of-the-art CNN, and finds in terms of compressing the most storage demanding dense connected layers, vector quantization methods have a clear gain over existing matrix factorization methods.

本文介绍 Gemma，一族基于 Gemini 模型所使用的研究与技术构建的轻量级 SOTA 开源模型，并全面评估模型的安全性与责任性，同时详细描述模型开发过程。This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models, and presents comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development.