研究发现 RLMs 能够成功处理超出模型上下文窗口长达两个数量级的输入，即便在较短 prompt 下，其质量也显著优于原生前沿 LLM 以及常见的长上下文与编程脚手架。It is found that RLMs can successfully process inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of vanilla frontier LLMs and common long-context and coding scaffolds.
论文
4 张论文卡片 · LLM 基础设施 · 方法 · 其他正式来源
论文信息
2. DualPath：打破 Agentic LLM 推理的存储带宽瓶颈
本文提出 DualPath，一种 inference 系统，通过引入 dual-path KV-Cache loading 打破瓶颈，并实现一条新的 storage-to-decode 路径：KV-Cache 先加载到 decode engine，再通过 compute network 上的 RDMA 高效转发至 prefill engine。DualPath is presented, an inference system that breaks this bottleneck by introducing dual-path KV-Cache loading and enables a novel storage-to-decode path, in which the KV-Cache is loaded into decoding engines and then efficiently transferred to prefill engines via RDMA over the compute network.
12. KVP：RL 驱动 KV Cache 驱逐策略
本文提出 KV Policy (KVP)，一种仅基于 key 与 value 向量、在预计算生成轨迹上训练的轻量级 per-head RL Agent 框架，证明学习预测未来 token 效用是自适应 KV cache 管理中强大且可扩展的范式。KV Policy (KVP), a framework of lightweight per-head RL agents trained on pre-computed generation traces using only key and value vectors, is introduced, demonstrating that learning to predict future token utility is a powerful and scalable paradigm for adaptive KV cache management.
Information-Aware KV Cache Compression for Long Reasoning
面向长推理的信息感知 KV Cache 压缩
本文从前瞻视角重新审视 token 重要性，提出衡量压缩 token 对未来上下文影响的新指标 Forward Influence，以及融合信息论信号的熵感知 KV cache 压缩框架 InfoKV。This paper revisits token importance from a forward-looking perspective and introduces Forward Influence, a metric that measures how compressed tokens affect future contexts, and proposes InfoKV, an entropy-aware KV cache compression framework that incorporates information-theoretic signals.