论文

本文综述了 Islamic LLMs 与可信 Islamic AI 这一新兴领域，并论证了仅具备阿拉伯语流利度不足以支撑 Islamic AI，进而提出面向抗幻觉 Islamic AI 系统的研究议程。This survey reviews the emerging field of Islamic LLMs and trustworthy Islamic AI, and argues that fluency in Arabic is not sufficient for Islamic AI, with a research agenda for hallucination-resistant Islamic AI systems.

AsymCache 是一个面向 LLM 推理的计算-延迟感知 KV cache 管理系统，将 cache 驻留决策与 GPU attention kernel 性能显式对齐，包含三个关键组件：用于高效处理非连续 KV 上下文的多段注意力（MSA）、联合优化命中率与位置感知重计算代价的 cache 淘汰策略，以及面向高硬件利用率的自适应分片调度器。AsymCache is proposed, a computation-latency-aware KV cache management system for LLM inference that explicitly aligns cache residency decisions with GPU attention kernel performance, including three key components: Multi-Segment Attention (MSA) for efficient non-contiguous KV context processing, a cache eviction policy that jointly optimizes hit rate and position-aware recomputation cost, and an adaptive chunking scheduler for high hardware utilization.

RTP-LLM 是一个面向工业级 LLM 部署的高性能推理引擎，已在 Alibaba Group 成功部署，服务超过 1 亿用户，通过集成设计解决根本性瓶颈。RTP-LLM is presented, a high-performance inference engine for industrial-scale LLM deployment, successfully deployed across Alibaba Group serving over 100 million users, and addresses fundamental bottlenecks through integrated design.

这篇立场论文认为，LLM 推理 serving 已超越通用启发式方法，如今需要数学优化与算法基础，并呼吁社区将 LLM serving 的算法设计视为一个新的研究前沿。This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic foundations, and calls on the community to recognize algorithmic design for LLM serving as a research frontier.

SwiftCache 是一个协同推理系统，使异构模型可在同一服务器内共享未充分利用的 GPU 内存与 NVLink 带宽，支持跨模型通过 NVLink 共享 KV cache，避免使用慢速 PCIe 传输。SwiftCache is a collaborative inference system that enables heterogeneous models to share underutilized GPU memory and NVLink bandwidth within a server, allowing cross-model KV cache sharing over NVLink and avoiding slow PCIe transfers.

本文对近期 KV cache 优化技术进行系统综述，将其归纳为五大方向：cache eviction、cache compression、混合内存方案、新型 attention 机制与组合策略，并指出自适应多阶段优化流水线是未来研究的重要方向。This paper provides a systematic review of recent KV cache optimization techniques, organizing them into five principal directions: cache eviction, cache compression, hybrid memory solutions, novel attention mechanisms, and combination strategies, and pointing toward adaptive, multi-stage optimization pipelines as a promising direction for future research.

研究发现 RLMs 能够成功处理超出模型上下文窗口长达两个数量级的输入，即便在较短 prompt 下，其质量也显著优于原生前沿 LLM 以及常见的长上下文与编程脚手架。It is found that RLMs can successfully process inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of vanilla frontier LLMs and common long-context and coding scaffolds.

本文提出了一种实证方法，用于研究基于 GPU 的 LLM 推理服务系统中的 software aging 问题，并提供了一个可复现的框架，开辟了 software aging 与 software rejuvenation 与 LLM serving 交叉方向的研究。This paper proposes an empirical methodology to study software aging in GPU-based LLM serving systems and provides a reproducible framework that opens a research direction at the intersection of the software aging and rejuvenation and LLM serving communities.

介绍 Albireo，一种并行推理系统：通过调度与 I/O 与计算及序列并行采样的重叠来压缩不可扩展部分，从而提高可达到的 TP 度 t，且不改变模型架构。Albireo is presented, a parallel inference system that raises the attainable TP degree t by shrinking the non-scalable portion via overlap of scheduling and I/O with compute and sequence-parallel sampling, without changing model architectures.

本工作提出一种新颖的混合精度 PTQ 策略，直接最小化整个模型的全局误差传播，而非孤立地处理逐层误差，并开发了一种新颖的联合优化方法，在统一搜索空间中同时学习结构化剪枝决策与混合精度量化策略。This work proposes a novel mixed-precision PTQ strategy that directly minimizes global error propagation across the entire model, rather than isolating layer-wise errors, and develops a novel joint optimization approach that simultaneously learns structural pruning decisions and mixed-precision quantization policies within a unified search space.

本工作借鉴线性模型的 state space model（SSM）视角，提出三项核心方法改进并组合形成更具表达力的递推结构：源自 SSM 离散化的递推式、用于更丰富状态追踪的复数值状态更新规则，以及在不增加 decode 延迟前提下提升模型性能的多输入多输出（MIMO）建模。This work introduces three core methodological improvements inspired by the state space model (SSM) viewpoint of linear models that combine to form a more expressive recurrence derived from SSM discretization, a complex-valued state update rule that enables richer state tracking, and a multi-input, multi-output (MIMO) formulation for better model performance without increasing decode latency.

Nemotron 3 Super 是 Nemotron 3 系列中首个采用 NVFP4 进行预训练的模型，借助 LatentMoE（一种同时优化精度 per FLOP 与精度 per parameter 的新型 Mixture-of-Experts 架构），并集成 MTP 层以通过原生 speculative decoding 加速推理。Nemotron 3 Super is the first model in the Nemotron 3 family to be pre-trained in NVFP4, leverage LatentMoE, a new Mixture-of-Experts architecture that optimizes for both accuracy per FLOP and accuracy per parameter, and include MTP layers for inference acceleration through native speculative decoding.

Tutti 是一种高效的 SSD-backed KV caching 方案，将 CPU 从 HBM 与 SSD 之间的关键数据与 I/O 控制路径中彻底移除，在提供近乎无限容量的同时，实现了与 DRAM-backed LMCache 几乎相当的 inference 性能。Tutti is an efficient SSD-backed KV caching solution that eliminates CPU intervention from the critical data and I/O control paths between HBM and SSDs, and achieves nearly the same inference performance as DRAM-backed LMCache, while providing almost infinite capacity.

本文提出了一种简单的 flow-control 框架，通过控制 prompt 加入 LLM 活跃集合的速率，实现更高的 token 与 request 吞吐量、更低的平均与尾部 latency，以及更稳定的 KV cache 利用率。A simple flow-control framework is proposed that controls the rate at which prompts join the active set in large language models and achieves higher token and request throughput, lower average and tail latency, and more stable KV cache utilization.

本文提出 DualPath，一种 inference 系统，通过引入 dual-path KV-Cache loading 打破瓶颈，并实现一条新的 storage-to-decode 路径：KV-Cache 先加载到 decode engine，再通过 compute network 上的 RDMA 高效转发至 prefill engine。DualPath is presented, an inference system that breaks this bottleneck by introducing dual-path KV-Cache loading and enables a novel storage-to-decode path, in which the KV-Cache is loaded into decoding engines and then efficiently transferred to prefill engines via RDMA over the compute network.

本文探讨容器化、微服务、动态调度等 Cloud Native 技术如何从根本上提升 LLM 推理服务，并展示 Cloud Native 系统在高需求场景下实现更高效资源分配、降低延迟与提升吞吐的能力。This article explores how Cloud Native technologies, such as containerization, microservices, and dynamic scheduling, can fundamentally improve LLM inference serving and demonstrates how a Cloud Native system enables more efficient resource allocation, reduces latency, and enhances throughput in high-demand scenarios.

本文首次对 vLLM 启动延迟进行了详细的性能表征，并构建了一个轻量级分析模型，能够针对给定硬件配置准确预测 vLLM 的启动延迟，为大规模推理环境中的资源规划提供了可操作的指导。This paper presents the first detailed performance characterization of vLLM startup latency and develops a lightweight analytical model that accurately predicts vLLM's startup latency for a given hardware configuration, providing actionable guidance for resource planning in large-scale inference environments.

NetKV，一种使用该 oracle 信息的 O(|D|) 每请求贪心策略，其层级排序被证明对过时遥测数据具有鲁棒性；并证明随着上下文长度增长，忽略网络项会使仅缓存感知的调度任意次优。NetKV, the O(|D|) per-request greedy that consumes this oracle, has tier rankings that are provably robust to stale telemetry, and it is proved that ignoring the network term renders cache-aware-only scheduling arbitrarily suboptimal as context length grows.

本工作提出 LMCACHE，首个也是目前最高效的开源 KV 缓存方案，可将现代 LLM 引擎生成的 KV 缓存从 GPU 显存中提取并存储，并跨引擎和查询共享。This work presents LMCACHE, the first and so far the most efficient open-source KV caching solution, which extracts and stores KV caches generated by modern LLM engines out of the GPU memory and shares them across engines and queries.

通过从概率分布的动态 nucleus 中采样文本，可在有效截断不可靠分布尾部的同时保持多样性，使生成文本更接近人类文本质量，在不牺牲流畅性与连贯性的前提下提升多样性。By sampling text from the dynamic nucleus of the probability distribution, which allows for diversity while effectively truncating the less reliable tail of the distribution, the resulting text better demonstrates the quality of human text, yielding enhanced diversity without sacrificing fluency and coherence.

本文介绍 Gemma，一族基于 Gemini 模型所使用的研究与技术构建的轻量级 SOTA 开源模型，并全面评估模型的安全性与责任性，同时详细描述模型开发过程。This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models, and presents comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development.

本工作研究图结构输入的特征学习技术，并在程序验证任务上取得 SOTA 性能，该任务需将子图与抽象数据结构进行匹配。This work studies feature learning techniques for graph-structured inputs and achieves state-of-the-art performance on a problem from program verification, in which subgraphs need to be matched to abstract data structures.

提出 MultiMedQA 基准，整合六个现有医学问答数据集（涵盖专业医学、研究与消费者查询）及一个全新的在线医学问题搜索数据集，并提出针对模型答案的人工评估框架，揭示了 LLM 在医学领域的潜在应用价值。MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, is presented and a human evaluation framework for model answers is proposed, suggesting the potential utility of LLMs in medicine.

本文给出 ICL 的形式化定义，厘清其与相关研究的联系，并梳理讨论训练策略、提示设计策略及相关分析等高级技术。This paper presents a formal definition of ICL and clarify its correlation to related studies, and organizes and discusses advanced techniques, including training strategies, prompt designing strategies, and related analysis.