AsymCache 是一个面向 LLM 推理的计算-延迟感知 KV cache 管理系统，将 cache 驻留决策与 GPU attention kernel 性能显式对齐，包含三个关键组件：用于高效处理非连续 KV 上下文的多段注意力（MSA）、联合优化命中率与位置感知重计算代价的 cache 淘汰策略，以及面向高硬件利用率的自适应分片调度器。AsymCache is proposed, a computation-latency-aware KV cache management system for LLM inference that explicitly aligns cache residency decisions with GPU attention kernel performance, including three key components: Multi-Segment Attention (MSA) for efficient non-contiguous KV context processing, a cache eviction policy that jointly optimizes hit rate and position-aware recomputation cost, and an adaptive chunking scheduler for high hardware utilization.
论文
4 张论文卡片 · LLM 基础设施 · 观点
5️⃣ Multi-Segment Attention · 分块位置感知KV驱逐 — arXiv:2606.02964（⭐⭐⭐ 新鲜 arXiv）
7️⃣ arXiv · Position Paper：LLM Serving 需要数学优化，而非仅靠启发式 ⭐⭐⭐⭐⭐ 学术前沿
这篇立场论文认为，LLM 推理 serving 已超越通用启发式方法，如今需要数学优化与算法基础，并呼吁社区将 LLM serving 的算法设计视为一个新的研究前沿。This position paper argues that LLM inference serving has outgrown generic heuristics and now demands mathematical optimization and algorithmic foundations, and calls on the community to recognize algorithmic design for LLM serving as a research frontier.
LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling
LoopCoder-v2：仅循环一次以实现高效测试时计算扩展
本文通过收益–成本视角研究 PLT 循环次数选择：额外循环可精炼表示，但 CLP 也会在每次循环边界引入位置错配，由此解释 PLT 在两次循环时趋于饱和的现象，并为循环次数选择提供诊断依据。This study studies PLT loop-count selection through a gain--cost view: an extra loop may refine representations, but CLP also introduces a positional mismatch at each loop boundary, explaining PLT's saturation at two loops and providing diagnostics for loop-count selection.
HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management
HiSparse：通过分层 KV 缓存管理扩展稀疏注意力解码
HiSparse 已合入上游 SGLang，并在 H200、B200 与 GH200 平台上针对 DSA、NSA、Quest 三类稀疏注意力家族进行评估：在长上下文负载下峰值生成吞吐提升最高达 4.7×，同时保持可比的 per-token 延迟，并降低高负载下的 time-to-first-token。HiSparse is merged into upstream SGLang and evaluated across three sparse-attention families (DSA, NSA, and Quest) on H200, B200, and GH200 platforms: it improves peak generation throughput by up to 4.7x on long-context workloads while preserving comparable per-token latency and reducing time-to-first-token at high load.