MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup

类型：arxiv
标识：2609.15126
链接：https://arxiv.org/abs/2609.15126
主分类：rag
形态：method
TLDR：Scaling large language models efficiently has motivated sparse capacity mechanisms such as Mixture-of-Experts and, more recently, conditional memory: token-indexed embedding tables that augment the backbone with cheap parametric lookups. Existing memory-embedding methods retrieve via a deterministic function of the surface form, which collapses different contextual senses of the same token (e.g., python the language vs. the animal) into a single fixed entry. We introduce Mixture of Memory Embeddings (MoME), a context-aware memory mechanism that replaces each token's single memory row with a mi
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-09-21-agent-rag-longcontext-candidates.json

2026-07-15

攻略 SAS: 通过端到端上下文排序实现稀疏注意力 · 干货攻略

2026-09-16

笔记 2026-09-18 flyP 精读：M3Exam 多模态记忆基准 + 2026 Agent Eval Stack

2026-09-18

笔记 ReMemR1: Look Back to Reason Forward (Revisitable Memory for Long-Context LLM Agents)

2026-06-12

笔记 Tom 文献雷达 · AI Agent / RAG / 长上下文 · 2026-08-03 晚间

2026-08-03

笔记 Tom 文献雷达 · Agent · RAG · Long Context · 2026-09-13 20:40

2026-09-13

arXiv

2609.15126

TLDR

Scaling large language models efficiently has motivated sparse capacity mechanisms such as Mixture-of-Experts and, more recently, conditional memory: token-indexed embedding tables that augment the backbone with cheap parametric lookups. Existing memory-embedding methods retrieve via a deterministic function of the surface form, which collapses different contextual senses of the same token (e.g., python the language vs. the animal) into a single fixed entry. We introduce Mixture of Memory Embeddings (MoME), a context-aware memory mechanism that replaces each token's single memory row with a mi

MoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup

相关 · 按语义匹配