Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning

类型：arxiv
标识：2609.35767
链接：https://arxiv.org/abs/2609.35767
主分类：multimodal
形态：method
TLDR：Unified multimodal models can both look at and render images, so in principle they can repair their own generations: diagnose what an image gets wrong, revise it, observe the result, and diagnose again. Whether a revision helps is known only after it is rendered, so the reflection text and the image generation must be learned jointly, over the whole loop. Supervised fine-tuning (SFT) on reflection trajectories gives a cold start but does not find the high-success repair paths, and naive RL that optimizes only the renderer or only one head leaves most of the gain untapped. We introduce UMM-Refl
待LLM分类：否
来源文件：
/inbox/tom/_candidates/2026-09-29-agent-rag-longcontext-candidates.json

2026-07-06

笔记 flyP 精读与批判 · VLA-Precision / ACoB（在线 RL for VLA 高精度操作）

2026-09-29

笔记 Cameron R. Wolfe (Deep Learning Focus) · RSS 摘要

2026-09-02

笔记 flyP 精读与反方审稿 · Agentic RL 综述 + GLM-5.2 开放 Agent 路线 · 2026-07-15 22:50

2026-07-15

笔记 Tom 文献雷达 · Agent + RAG + 长上下文 · 2026-08-27 20:40

2026-08-27

笔记批判性精读 · OraRL "Annotations as Rollouts" `arXiv:2608.20492` · 视频 MLLM 强化学习样本效率新范式

2026-08-27

arXiv

2609.35767

TLDR

Unified multimodal models can both look at and render images, so in principle they can repair their own generations: diagnose what an image gets wrong, revise it, observe the result, and diagnose again. Whether a revision helps is known only after it is rendered, so the reflection text and the image generation must be learned jointly, over the whole loop. Supervised fine-tuning (SFT) on reflection trajectories gives a cold start but does not find the high-success repair paths, and naive RL that optimizes only the renderer or only one head leaves most of the gain untapped. We introduce UMM-Refl

arXiv ↗ 中文全文

Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning

相关 · 按语义匹配