内容库 / 论文

arXiv:2608.02703 · LLM 基础设施

ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

ARCHead: 大语言模型输出头的激活度量残差校正

ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads

类型：arxiv
标识：2608.02703
链接：https://arxiv.org/abs/2608.02703
主分类：llm-infra
形态：method
被引：0
被引来源：Semantic Scholar
S2被引：0
影响力被引：0
TLDR：ARCHead is presented, a packed LM-head compressor that combines a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric to complements block quantizers by compressing the large output projection they can leave untouched.
待LLM分类：否
标题中文：ARCHead: 大语言模型输出头的激活度量残差校正
TLDR中文：提出 ARCHead，一种紧凑型 LM-head 压缩器，结合量化低秩核心、组级 INT4 残差以及基于激活导出度量拟合的低秩校正，在 block 量化器无法触及的大输出投影上实现互补压缩。
来源文件：
/inbox/tom/_candidates/2026-08-06-agent-rag-longcontext-candidates.json
[S2 enrich]

相关 · 按语义匹配

仓库 leon2k2k2k/llm-feedback-alignment

笔记精读：Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects

笔记 2026-07-04 上午多模态对照精读 · Seeker（文本→像素的长上下文 MLLM）

仓库 griver/Q-RAG

笔记 MMProLong:长上下文视觉语言模型的有效续训练(精读 · flyP)

笔记 flyP 精读与批判 · 2026-07-29 22:50（v2 覆盖）

arXiv

2608.02703

TLDR

提出 ARCHead，一种紧凑型 LM-head 压缩器，结合量化低秩核心、组级 INT4 残差以及基于激活导出度量拟合的低秩校正，在 block 量化器无法触及的大输出投影上实现互补压缩。ARCHead is presented, a packed LM-head compressor that combines a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric to complements block quantizers by compressing the large output projection they can leave untouched.

被引 · S2

0

S2 0

影响力 0

相关解读

arXiv ↗ 中文全文