研究发现,recall@k 并非已部署 KB-VQA 的正确评价指标,且弥合差距需要 reader 侧介入;本文首次对多模态 KB-VQA 中 reader 侧位置依赖性进行了受控探查,设计了一种 gold-position 协议——在问题提示中仅改变 gold passage 所在的槽位。The first controlled probe of reader-side position dependence in multimodal KB-VQA is designed, a gold-position protocol in which only the gold passage's prompt slot varies within question, indicating that recall@k is the wrong metric for deployed KB-VQA and that the remaining headroom sits on the reader side.
论文
19 张论文卡片 · 多模态 · 观点
提出 Decoupled Block Attention,在保留共享 video-query 上下文访问的同时消除跨 box 依赖,并结合用于时间边界与空间几何的 localization-aware policy optimization;并行 tube generation 被证明是视频中 autoregressive 定位的一种高效替代方案。Decoupled Block Attention is introduced, which preserves access to shared video-query context while eliminating cross-box dependencies, together with localization-aware policy optimization for temporal boundaries and spatial geometry, and parallel tube generation is shown to be an efficient and effective alternative to autoregressive localization in videos.
我们报告了一项历时数月的工程项目,用于构建 Sophea,一个生产级希腊语-英语双语自动语音识别系统。我们依据九个生产门控评估该系统,涵盖希腊语和英语词错误率、语言识别及非语音音频的幻觉问题。经过二十三次训练迭代和两种模型架构,没有任何训练数据组合能同时通过全部九个门控。满足希腊语噪声环境目标需要约1,500步密集领域暴露,而保持英语语言识别仅能容忍约2%We report a multi-month engineering program to build Sophea, a production bilingual Greek-English automatic speech recognition system. We evaluate the system against nine production gates covering Greek and English word error rate, language identification, and hallucinations on non-speech audio. Across twenty-three training iterations and two model architectures, no training-data composition passed all nine gates simultaneously. Meeting the Greek noisy-environment target required about 1,500 steps of dense domain exposure, while preserving English language identification tolerated only about 2
PhysStream 是一个用于物理驱动图像到视频合成的自回归模型,通过引入结构化场景记忆并支持基于稀疏速度增量信号的细粒度运动控制(编码物理量),使模型能够学习底层动力学。PhysStream is an autoregressive model for physics-grounded image-to-video synthesis that incorporates structured scene memory and supports fine-grained motion control via sparse velocity-increment signals that encode physical quantities, letting the model learn the underlying dynamics.
Register被实现为专用的固定位置 token,其连续的隐藏状态被训练用于跨生成块承载推理进度,对有界代码生成尤其有效,因为正确程序通常跨越多个块。Registers are implemented as dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks, which are especially effective for bounded code generation, where correct programs usually span several chunks.
本文提出物理秩一致性 (PRC) 来衡量 tokenization 在重建后保留局部物理距离排序的程度,并提出 ActionPiece,通过对表示学习和量化的联合监督来保留物理动作关系。This work introduces physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction, and presents ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization.
基于信噪分解与候选集近似的 IER(信息效率比),可基于 IER 及其与现有效用分数的组合进行 token 选择,同时保留采样的反向 KL 训练目标。An information-efficiency ratio (IER) based on a signal-to-noise decomposition and a candidate-set approximation enables token selection based on IER and its combination with existing usefulness scores, while retaining the sampled reverse-KL training objective.
提出 Mira-Scene,一种组合式 3D 场景重建框架,将稀疏姿态回归替换为密集有界对应恢复,并引入多模态扩散 Transformer 联合生成物体几何与 CCM,使用模态专精的专家流配合共享注意力与位置编码以促进几何-布局一致性。Mira-Scene is presented, a compositional 3D scene reconstruction framework that replaces sparse pose regression with dense, bounded correspondence recovery and introduces a multimodal diffusion transformer that jointly generates object geometry and CCMs, using modality-specific expert streams with shared attention and positional encoding to promote geometry-layout consistency.
本文提出 Uranus,一个以关节轨迹为条件的自回归扩散模型为核心的数据驱动机器人仿真器,为不同机器人本体与相机配置下的同步多视图生成提供统一接口。This work presents Uranus, a data-driven robot simulator built around a joint-trajectory-conditioned autoregressive diffusion model, providing a unified interface for synchronized multi-view generation across diverse robot embodiments and camera configurations.
扩散模型能生成惊艳的图像与视频,但在组合性细节(物体数量、属性绑定、空间关系、时序动作等)上仍难以严格遵循 prompt。提升 prompt 满足度的常见做法是在测试时通过 Best-of-N 采样投入更多算力,但最终样本的选择是固定的:Best-of-N 只能从已完成的输出中挑选,无法在有潜力的轨迹失败前予以修复。我们提出 PreviewDiff,一种无需训练、基于测试时搜索的方法,将扩散采样转化为Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts, attribute binding, spatial relations, and temporally grounded actions. A common way to improve prompt satisfaction is to spend more compute at test time through Best-of-N sampling, but final-sample selection is fixed. Best-of-N can only choose among completed outputs and cannot repair a promising trajectory before it fails. We introduce PreviewDiff, a training-free test-time search method that turns diffusion samplin
转移层面的结果表明,state adaptation 选择性而非统一地应用时最为有效,且 adaptation 的价值取决于策略反转的频率与幅度。The transition-level results suggest that state adaptation is most useful when applied selectively rather than uniformly, and that adaptation value depends on both the frequency and magnitude of strategy reversals.
本文论证了稀疏组合监督与动宾学习的不对称性会助长物体驱动的捷径学习,并指出减少捷径诊断可提升组合泛化能力。This work argues that sparse compositional supervision and verb-object learning asymmetry can promote object-driven shortcut learning and reduces shortcut diagnostics and consequently improves compositional generalization.
本文从玩家动作控制、游戏状态动态、状态-观测持久性与实时交互生成四个维度审视交互式游戏世界建模,并针对《Black Myth: Wukong》提出可扩展的数据引擎,采集超过 90 小时的游戏画面作为状态感知型游戏世界建模的资源。This paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation, and presents a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay as a resource for state-aware game world modeling.
提出了ShotPlan,一个基于视频扩散基础模型构建的、用于显式多镜头电影级视频生成的框架,显著优于现有的电影级视频生成方法,提供更灵活的镜头管理和更强的跨镜头一致性。ShotPlan is proposed, a framework for explicit multi-shot cinematic video generation built upon a video diffusion foundation model that significantly outperforms existing cinematic video generation methods, offering more flexible shot management and stronger inter-shot consistency.
提出结构化动态模型(SDM),通过未来特征预测,显式地将时间变化的主导来源与残差动态分离开来,而非使用单一纠缠的隐变量或非结构化的、空间密集的转移 token 来表示视频变化。The Structured Dynamics Model (SDM) is proposed, which explicitly separates the dominant source of temporal change from residual dynamics through future-feature prediction, rather than representing video change with a single entangled latent or with unstructured, spatially dense transition tokens.
将 Positive--Direction Matching (PDM)——一种分支感知的 OPD 目标,分别约束正预测方向与 CFG 条件方向——引入 dense-to-sparse 视频控制;由于朴素的 guided matching 对推理 guidance 尺度极为敏感,分支感知监督可实现更鲁棒、更有效的知识迁移。Positive--Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction, is introduced to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.
提出一种自适应推理时引导框架,利用VLA Latents上的强化学习,发现推理时引导在成功与失败状态下遵循根本不同的scaling laws:动作多样性在基础VLA可能失败时最为有益,但在成功可能性高时可能不必要地扰动已准确的动作。This work introduces an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents, and discovers that inference-time steering follows fundamentally different scaling laws under success and failure states, revealing that action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely.
EffectLearner 是一个语义推理增强框架,结合基于 VLM 的 Object-Effect Reasoner 与基于 DiT 的 Video Eraser,在 EffectWorld-Eval 和具有挑战性的 EffectWorld-Wild 上均取得明显优势,证明其能在复杂真实场景中实现高质量的视频物体擦除。EffectLearner is proposed, a semantic-reasoning-enhanced framework that combines a VLM-based Object-Effect Reasoner with a DiT-based Video Eraser that achieves clear advantages on both EffectWorld-Eval and the challenging EffectWorld-Wild, demonstrating its ability to deliver high-quality video object removal in complex real-world scenes.
在客观与主观指标上的重建和生成结果匹配或超越前沿开源 tokenizer,包括 Wan-2.2、HunyuanVideo-1.5、FLUX.2、MovieGen、StableAudio 和 MMAudio 的 VAE。It is demonstrated that reconstruction and generation results on objective and subjective metrics matches or surpasses frontier opensource tokenizers, such as VAEs from Wan-2.2, HunyuanVideo-1.5, FLUX.2, MovieGen, StableAudio and MMAudio.