Memorizon: Training World Models Beyond Their Context Window

• 类型：arxiv
• 标识：2610.00544
• 链接：https://arxiv.org/abs/2610.00544
• 主分类：engineering
• 形态：method
• TLDR：Streaming world models should render a place consistently across repeated visits. Directly supervising such revisits requires training samples that capture both visits, often spanning minutes. Yet dense attention over the full span incurs quadratic costs, making long-span supervision expensive. Memorizon breaks this coupling: long spans are needed for supervision, but not for attention, since the two visits can share a forward pass without including every intervening frame. A training sample covers a span of any length but is scored only on its last k chunks. Instead of tokenizing the history
• 副分类：multimodal
• 待LLM分类：否
• 来源文件：
• /inbox/tom/_candidates/2026-10-02-agent-rag-longcontext-candidates.json