Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention

• 类型：arxiv
• 标识：2609.24797
• 链接：https://arxiv.org/abs/2609.24797
• 主分类：engineering
• 形态：method
• TLDR：Linear RNNs based on the delta-rule enable efficient sequence modeling, but their linear updates with a low-rank correction constrain their expressivity. Prior work has shown that composing two delta-rule transitions in a single recurrent update can model a 2D rotation, but this increases the rank and the cost of the updates compared to a single transition. We show that Kimi Delta Attention (KDA) can realize 2D rotations by combining a single delta-rule transformation with a second reflection supplied by its channel-wise gate. This requires extending the parameter ranges of KDA by combining tw
• 待LLM分类：是
• 来源文件：
• /inbox/tom/_candidates/2026-09-22-agent-rag-longcontext-candidates.json