论文

AsymCache 是一个面向 LLM 推理的计算-延迟感知 KV cache 管理系统，将 cache 驻留决策与 GPU attention kernel 性能显式对齐，包含三个关键组件：用于高效处理非连续 KV 上下文的多段注意力（MSA）、联合优化命中率与位置感知重计算代价的 cache 淘汰策略，以及面向高硬件利用率的自适应分片调度器。AsymCache is proposed, a computation-latency-aware KV cache management system for LLM inference that explicitly aligns cache residency decisions with GPU attention kernel performance, including three key components: Multi-Segment Attention (MSA) for efficient non-contiguous KV context processing, a cache eviction policy that jointly optimizes hit rate and position-aware recomputation cost, and an adaptive chunking scheduler for high hardware utilization.

这篇立场论文认为，LLM 推理 serving 已超越通用启发式方法，如今需要数学优化与算法基础，并呼吁社区将 LLM serving 的算法设计视为一个新的研究前沿。This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic foundations, and calls on the community to recognize algorithmic design for LLM serving as a research frontier.

冻结一个大规模评测集合，并尝试对其历史断言进行 replay，以使这一原本隐式的推理步骤变得显式且可执行。A large evaluation collection is frozen and an attempt to replay its historical claims is made to make this otherwise implicit inference step explicit and executable.

揭示了复用造成的质量损失以及有效修复方法均取决于 LLM（即便在两个 8B 模型之间也是如此），并提供添加新方法的通用接口与交互式 leaderboardBoth the quality loss from reuse and which repairs help depend on the LLM, even between two 8B models are introduced, as well as a common interface for adding new methods and an interactive leaderboard.

该工作提出了分组值注意力（Grouped Value Attention, GVA），通过存储分组值并在推理时利用可学习的线性映射重构内容键，该映射可被吸收进 query 中，从而在预期解码路径中无需显式物化内容键。This work introduces Grouped Value Attention (GVA), which stores grouped values and reconstructs content keys with a learned linear map at inference, which can be absorbed into the query, eliminating the need to materialize content keys in the intended decode path.

本文论证线性表征假设不是单一假设，而是一组通过表征等价性加以区分的命题族；该框架厘清了假设如何随度量、读取点与分析阶段变化，并被用于审计常见表征量与近期可解释性分析。It is argued that the Linear Representation Hypothesis is not one hypothesis but a family of claims distinguished by representation equivalence, and this framework clarifies how assumptions can change across metrics, reading points, and analysis stages, and is used to audit common representation quantities and recent interpretability analyses.

Boxoffice 是一个程序化生成评测数据集的工具，用以覆盖具有挑战性的 KV cache 复用模式；研究表明现有数据集并未展现出充分评估此类技术所需的复用动态特性。Boxoffice, a tool that programmatically generates evaluation datasets that exercise challenging KV cache reuse patterns, is introduced and it is shown that existing datasets do not exhibit the reuse dynamics needed to thoroughly evaluate such techniques.

本文揭示使用 chunked KV-cache 压缩的模型中存在一种系统性不对称：相同信息在一个阶段易于检索，在另一阶段却难以获取，暴露出平均 benchmark 分数可能掩盖的周期性薄弱点。A systematic asymmetry in models using chunked KV-cache compression is uncovered: the same information can be easy to retrieve at one phase and difficult at another, revealing periodic weak spots that average benchmark scores can conceal.

本文通过收益–成本视角研究 PLT 循环次数选择：额外循环可精炼表示，但 CLP 也会在每次循环边界引入位置错配，由此解释 PLT 在两次循环时趋于饱和的现象，并为循环次数选择提供诊断依据。This study studies PLT loop-count selection through a gain--cost view: an extra loop may refine representations, but CLP also introduces a positional mismatch at each loop boundary, explaining PLT's saturation at two loops and providing diagnostics for loop-count selection.

HiSparse 已合入上游 SGLang，并在 H200、B200 与 GH200 平台上针对 DSA、NSA、Quest 三类稀疏注意力家族进行评估：在长上下文负载下峰值生成吞吐提升最高达 4.7×，同时保持可比的 per-token 延迟，并降低高负载下的 time-to-first-token。HiSparse is merged into upstream SGLang and evaluated across three sparse-attention families (DSA, NSA, and Quest) on H200, B200, and GH200 platforms: it improves peak generation throughput by up to 4.7x on long-context workloads while preserving comparable per-token latency and reducing time-to-first-token at high load.