Constitutional Midtraining: Content Presence Drives Alignment Gains

  • 类型：arxiv
  • 标识：2607.26654
  • 链接：https://arxiv.org/abs/2607.26654
  • 主分类：risk
  • 形态：method
  • 被引：0
  • 被引来源：Semantic Scholar + OpenAlex
  • S2被引：0
  • OpenAlex被引：0
  • 影响力被引：0
  • TLDR：Post-training alignment is often shallow, eroding under fine-tuning. Whether midtraining interventions, cleanly isolated from post-training, can produce durable alignment remains untested. We test this via constitutional midtraining: inserting principled, values-based content into midtraining against a replay-only control at 120B scale. Our 394M-token constitutional corpus, built from Anthropic's Constitution, uses a 2x2 factorial design (curriculum ordering x deliberative reasoning) to produce four constitutionally midtrained conditions plus a control, evaluated on self-generated and establis
  • OpenAlex ID：W7171813301
  • OpenAlex DOI：10.48550/arxiv.2607.26654
  • DOI：10.48550/arxiv.2607.26654
  • DOI来源：OpenAlex
  • 开放获取：green
  • 开放获取链接：https://doi.org/10.48550/arxiv.2607.26654
  • OpenAlex更新：2026-08-26
  • 副分类：engineering
  • 待LLM分类：否
  • 标题中文：宪法式中训练：内容存在驱动对齐收益
  • TLDR中文：训练后对齐往往较浅，会在微调中被侵蚀。而中训练干预能否在干净隔离于训练后的情况下产生持久对齐，此前未经检验。我们通过宪法式中训练来测试：在 120B 规模上，插入基于原则与价值观的内容，与仅做回放的对照组进行对比。我们基于 Anthropic 的 Constitution 构建了 394M token 的宪法语料，并采用 2×2 析因设计（课程顺序 × 审慎推理），形成四种宪法式中训练条件与一组对照，随后在自生成与既有...
  • 来源文件：
  • /inbox/tom/_candidates/2026-08-04-rag-retrieval-reranking-candidates.json
  • /inbox/tom/_candidates/2026-08-04-agent-rag-longcontext-candidates.json
  • [S2 enrich]
  • [OpenAlex backfill]