研究库 论文知识库

内容库 / 论文

arXiv:2610.05842 · 工程化

HLA: Expressive Hybrid Linear Attention via Chunk-Wise Dynamic Mixing

HLA: Expressive Hybrid Linear Attention via Chunk-Wise Dynamic Mixing

类型：arxiv
标识：2610.05842
链接：http://arxiv.org/abs/2610.05842v1
主分类：engineering
形态：method
TLDR：Linear attention enables efficient long-context autoregressive decoding by compressing history into recurrent states, but this compression can make selective access to sparse and distant information difficult. Existing chunk-based extensions increase memory capacity, yet learned chunk-mixing coefficients may remain fixed with respect to input content and therefore cannot adapt historical access to each query. We introduce \emph{Hybrid Linear Attention} (HLA), a query-dependent chunk-level attention mechanism for Gated DeltaNet (GDN). HLA represents each completed chunk as an exact affine state
待LLM分类：是
来源文件：
/inbox/tom/_candidates/2026-10-06-agent-rag-longcontext-candidates.json

相关 · 按语义匹配

笔记 Tom 文献雷达 · AI Agent / RAG / 长上下文 · 2026-07-11 下午版

笔记 PagedAttention 2.0 推理优化 · 2026-07-28

攻略 SAS：端到端优化上下文排名的稀疏注意力机制 · 干货攻略

仓库 fla-org/flash-linear-attention

笔记 Tom 文献雷达 · Agent · RAG · 长上下文 · 2026-10-06

笔记 Cool Papers · cs.CL (papers.cool) · RSS 摘要

arXiv

2610.05842

TLDR

Linear attention enables efficient long-context autoregressive decoding by compressing history into recurrent states, but this compression can make selective access to sparse and distant information difficult. Existing chunk-based extensions increase memory capacity, yet learned chunk-mixing coefficients may remain fixed with respect to input content and therefore cannot adapt historical access to each query. We introduce \emph{Hybrid Linear Attention} (HLA), a query-dependent chunk-level attention mechanism for Gated DeltaNet (GDN). HLA represents each completed chunk as an exact affine state

arXiv ↗ 中文全文