MemSFT: Mitigating Alignment Tax with an External Parametric Memory

• 类型：arxiv
• 标识：2607.25614
• 链接：https://arxiv.org/abs/2607.25614
• 主分类：risk
• 形态：method
• 被引：1
• 被引来源：Semantic Scholar
• S2被引：1
• 影响力被引：0
• TLDR：Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting and substantially degrade performance on general tasks. We propose MemSFT, which mitigates the alignment tax by decoupling domain specialization from backbone parameter updates through a plug-and-play parametric memory. The memory is trained to imitate the behavior of a non-parametric retriever operating over domain data, thereby memorizing knowledge and patterns that would otherwise be accessed through retrieval. Once trained on
• 副分类：rag
• 待LLM分类：否
• 标题中文：标题 -> 标题中文：MemSFT：借助外部参数化记忆缓解对齐税
• TLDR中文：将 LLM 适配到专用领域常会带来对齐税：针对领域特定任务进行微调会导致灾难性遗忘，并显著降低在通用任务上的表现。我们提出 MemSFT，通过将领域专业化与主干参数更新解耦，以即插即用的参数化记忆来缓解对齐税。该记忆被训练为模仿在领域数据上运作的非参数化检索器，从而记住原本需通过检索获取的知识与模式。一旦在
• 来源文件：
• /inbox/tom/_candidates/2026-08-05-agent-rag-longcontext-candidates.json
• [S2 enrich]